Skip to Content
avatar image
Former Member

Problem with converting a floating-point value into a long value

Hi,

i encountert a problem with the conversion of a floating-point value into a simple long value. It seems like it only works correct with a decimal value as well as a constant.

eg.

long ( 99.8 ) = 100

which is totally fine,but the following seems wrong to me

DOUBLE ldbl_value = 99.8

long ( ldbl_value ) = 99

Why do i get a different result with that code?

What do you think of that?

I enclosed an image of a list, which showes how the conversion is creating different results, depending on the input type.

Best Regards

TimKp

Add comment
10|10000 characters needed characters exceeded

  • Follow
  • Get RSS Feed

1 Answer

  • avatar image
    Former Member
    Oct 28, 2015 at 05:04 PM

    Hi Tim;

    AFAIK: Long and INT are "whole" numbers and the decimal places are either truncated or rounded depending on the operations being performed in converting the decimal number. If you want to use floating-point then I would recommend using the REAL data type. That should preserve the decimals.

    Regards ... Chris

    Add comment
    10|10000 characters needed characters exceeded

    • Former Member Former Member

      Converting to doubles has always been a mapping to the long part of the number.

      It is actually the decimal result that is wrong and is a bug but that could have been in the product for more than a decade.

      Long() is similar to Int() and if you look to the Int function then it is defined as:

      Determines the largest whole number less than or equal to a number.

      Syntax

      Int ( n )

      If you wanted it to round then explicitly use the round function.