Is DECIMAL(p,s) precision limit really 38 or rather 34 ?

[sebflaesch]

The documentation states that DECIMAL(p,s) can have p<=38.
In reality, the exact max precision is 34 significant digits.
Otherwise, how do you explain:

create local temporary table #t1 ( pk int, d1 decimal(38,0) );
insert into #t1 values ( 101, 12345678901234567890123456789012345 );
select * from #t1 where d1 =  12345678901234567890123456789012345 or d1 =  1234567890123456789012345678901235 ; 

Returns:

0 rows affected (overall time 1754 usec; server time 353 usec)
1 row affected (overall time 2084 usec; server time 329 usec)
PK,D1
101,12345678901234567890123456789012350
1 row selected (overall time 3331 usec; server time 130 usec)

Can someone explain / confirm?
Several questions posted here an not much feedback... someone there?

[lbreddemann · Accepted solution]

Hmm... this one looks tricky and could be a bug.

Tracing the actual output of the query on different levels (JDBC, server-side SQL trace) shows that the result decimal is always rounded to the 34 precision (i.e. ending in ...12350 instead of ...12345). Internally, though, processing is not affected by this "rounding". So filtering/matching values works with the full precision and feeding the data into functions also works with the full precision.

Example 1:

select pk, to_bigint(d1) 
from t1 where d1 =  12345678901234567890123456789012345;

Started: 2017-09-18 14:34:43
Could not execute 'select pk, to_bigint(d1) from t1 where d1 = 12345678901234567890123456789012345' in 5 ms 283 µs . 
[314]: numeric overflow:cannot convert to BigInt type: 12345678901234567890123456789012345 at function to_bigint() (at pos 23) 

>>> FULL PRECISION IN ERROR MESSAGE

Example 2:

select pk, to_char(d1) from t1;
PK  TO_CHAR(D1)                       
101 12345678901234567890123456789012345

>>> FULL PRECISION AFTER CONVERSION TO CHAR

I just wanted to finish this comment off by writing that this should be checked by HANA support when I found SAP Knowledgebase article https://launchpad.support.sap.com/#/notes/2410312 (2410312 - Tail bits are set to zero for big decimal in SAP HANA). This KBA basically describes the same phenomenon and declares it as "not a bug".

So, while HANA (both HANA 1 and HANA 2) handle decimal internally with the full precision of 38 bits, the direct output is limited to 34 bit.

cheers,

Lars

[sebflaesch · Accepted solution]

Thank you Lars,

I understand the limit comes from the SAP Hana clients.

Seb

[lbreddemann · Accepted solution]

Not quite. The limit comes from how HANA serves the server side part of the client/server communication. So, all clients always only see the capped values.

[sebflaesch · Accepted solution]

Ok... anyway, from a user point of view:

• If it's a bug, it should be fixed rapidly because it's critical (data loss).
• If it's a known and accepted limitation, it should be documented (and server should return a warning when creating columns with DECIMAL(p>34[,s]) !

Seb

[lbreddemann · Accepted solution]

Yep, I agree with that.

[lbreddemann · Accepted solution]

FYI I opened an internal bug for this. So, let's see what the HANA devs do with this.

[sebflaesch · Accepted solution]

Ok Thanks!

[Taesuk]

As Lars mentioned, internally all data is treated as Decimal(38,0). Issue is when streaming the data to the client where it was rounded to 34 precision. This planned to be resolved for the next HANA release which is HANA 2.0 SPS03.