Graphical view - Decimal precision is lost in calculated field

[former_member372069]

Hi Experts,

We are replicating an ABAP report using a calculation view where we do certain calculations (decimal precision is very crucial). Along this process it seems the the decimal precision is lost. Say we have to compute the following:

14000/1.12*0.99625

NOTE: The last 2 operand are default. Only the first operand (14000) value may change via input parameter.

The result for this calculation is 12453.12500000000 whereas the expected value is 12453.124999999998.

This is the actual process and the value we derived between ABAP and HANA

1.) Calculate value as <input parameter>/1.12*0.99625

Example: 14000/1.12*0.99625

ABAP = 12453.124999999998

SAP = 12453.12500000000

2.) Get 100th decimal place

ABAP = 12453.124

SAP = 12453.125

3.) Round up to 2 decimal place

ABAP = 12453.12

SAP = 12453.13

As you notice, due to the precision we will be getting variance.

I already tried a number of combination of data type and conversion function but to no avail.

Can someone please enlighten me on this behavior and perhaps suggest fixes/workaround?

Regards,

Rod Castro

[FCI]

Hi Rod,

This seems to do the trick:

Regards,

Frederic

[former_member372069]

Thanks, Frederick. That was quick.

Initially, yes it works for this sample data. Let me just run a regression test to confirm it works for all data set.

[ArunaShibane]

I tried a hundred solutions and this one miraculously worked ! Thanks Frederick.

[sreevatsa_adiraju]

also you can try like below, keep same decimals as Frederick mentioned

(14000/1.12*0.99625)*1.000000000000

keep same number of ZERO's, check the data in Raw data.