Skip to Content
0
Former Member
May 30, 2007 at 02:38 PM

Quirk with Arithmetic MUL function

77 Views

All,

In my source file I've got 2 fields (sf1, sf2) declared as type xsd:string. My requirement is to multiply those two fields and map the value to my destination field (df1) which is also of type xsd:string.

What I've noticed is if I have, for instance, sf1=3427.00 and sf2=97.0 then df1 is equal 332419.000 (we also send the result through a FormatNumber routine to get the 3 decimal places). In this scenario we see no issue.

The problem appears when we use larger numbers. An example we've seen is if sf1=41041.00 and sf2=817.0. The answer SHOULD be 33530497.000. What it actually does is calculate the value as 3.3530496E7. When that gets mapped to df1 it ends up getting set to 33530496.000 which is off by 1. We have other examples where the value is off by a little more than 1.

My question are:

1) Can anyone explain to me why Java/XI calculates this way (if it's not a bug) and,

2) does anyone have an idea as to how I would resolve this issue?

We are on SP15.

Thanks!