Transform JSON with special characters using /ui2/cl_json=>deserialize not working

[former_member324825]

Hi Experts,

I am consuming data from a webservice using CL_HTTP_CLIENTSPAN=>create_by_destination which works perfectly fine. Afterwhich, I am using /ui2/cl_json=>deserialize to transform the json values to my internal table. Now, I am receiving data with non english characters such as é(\U00E9) and è(\u00E8) with their respective codes and unable to translate them as characters.

I also tried using cl_fdt_json=>json_to_data and all possible approaches that were provided by our experts in our SAP community but luck was not my side.

Hoping for your answers!

Regards,

Cliff

[alexey_arseniev · Accepted solution]

Hello Cliff,

yes, tomas.buryanek is right. /UI2/CL_JSON is not intended to do the decoding of HTML entities into Unicode characters.

You need to pass the JSON string through unescapement method first and then run deserialize. Unfortunately, it seems that there is no standard ABAP API for HTML unescapement 😞 .

The only way I see, based on the suggestion of Tomas:

1) Collect all Unicode characters in JSON

FIND ALL OCCURRENCES OF REGEX '\\u([0-9,a-z]{4})' IN json RESULTS matches.  

2) Loop over matches and replace substrings in the json with a value you get from cl_abap_conv_in_ce=>uccp as a result of the conversion of the first submatch (group) in the result.

3) Run /ui2/cl_json=>deserialize on processed JSON string.

BR, Alexey.

[former_member324825 · Accepted solution]

Thank you all for your comments. My issue was solved and my approach was very much similar to your solution, alexey.arseniev. I also got your suggestion of using the REGEX. I just tweaked the code and used \\u([0-9,a-z]{4})+ instead.

[Tomas_Buryanek]

Hello,
JSON deserializing usually does not "unescape" escaped unicode characters. You will have to do it yourself before or after deserializing JSON. You can try for example (maybe there are better ways) this method for converting hexa representation of unicode character:

cl_abap_conv_in_ce=>uccp

[Sandra_Rossi]

That's a bug of /UI2/CL_JSON, it's corrected by the SAP note 2382783 - /UI2/CL_JSON Corrections.

Excerpt:

Symptom
Fixed:
1) Unescape of symbol '\' on JSON deserialization does not work
...