Skip to Content

Graphical view - Decimal precision is lost in calculated field

Hi Experts,

We are replicating an ABAP report using a calculation view where we do certain calculations (decimal precision is very crucial). Along this process it seems the the decimal precision is lost. Say we have to compute the following:

14000/1.12*0.99625

NOTE: The last 2 operand are default. Only the first operand (14000) value may change via input parameter.

The result for this calculation is 12453.12500000000 whereas the expected value is 12453.124999999998.

This is the actual process and the value we derived between ABAP and HANA

1.) Calculate value as <input parameter>/1.12*0.99625

Example: 14000/1.12*0.99625

ABAP = 12453.124999999998

SAP = 12453.12500000000

2.) Get 100th decimal place

ABAP = 12453.124

SAP = 12453.125

3.) Round up to 2 decimal place

ABAP = 12453.12

SAP = 12453.13

As you notice, due to the precision we will be getting variance.

I already tried a number of combination of data type and conversion function but to no avail.

Can someone please enlighten me on this behavior and perhaps suggest fixes/workaround?

Regards,

Rod Castro

Add comment
10|10000 characters needed characters exceeded

  • Follow
  • Get RSS Feed

2 Answers

  • Mar 01 at 11:06 AM

    Hi Rod,

    This seems to do the trick:

    Regards,

    Frederic


    formula.jpg (35.9 kB)
    result.jpg (11.0 kB)
    formula.jpg (38.3 kB)
    Add comment
    10|10000 characters needed characters exceeded

    • Thanks, Frederick. That was quick.

      Initially, yes it works for this sample data. Let me just run a regression test to confirm it works for all data set.

  • Mar 01 at 02:17 PM

    also you can try like below, keep same decimals as Frederick mentioned

    (14000/1.12*0.99625)*1.000000000000

    keep same number of ZERO's, check the data in Raw data.

    Add comment
    10|10000 characters needed characters exceeded