Error with XML file in UTF-8 (special characters)

[Former Member]

We are needing to send an XML file from SAP to an external system in UTF-8 format. The ABAP code we are currently using works correctly as long as there are no special characters or accent marks within the XML content. However, whenever the XML file contains a special character or accented character (such as an e-acute, é), then the receiving system returns to us an error saying that they were unable to parse the file successfully. Therefore, we think that perhaps the file is not actually being sent in UTF-8 format, even though we specify in the XML header that 'encoding="UTF-8" ' and we set a header field of 'Content-Type=text/xml; charset=utf-8'.

Here is a description of what we are currently doing in our ABAP function module:

We first concatenate all of the XML into a variable of type STRING (named p_xml_out below). Then we create the HTTP client, set several header fields (including Content-Type=text/xml; charset=utf-8), call the method SET_CDATA to prepare to send the STRING of data, and finally call the method to SEND the data.

When we realized that we were receiving the Parse error message back from the external system, we then tried an alternative approach of converting the STRING to type XSTRING and specifying that the encoding should be 'UTF-8'. Then we used method SET_DATA (instead of SET_CDATA) before calling the SEND method.

However, this code results in the exact same Parse error. We are afraid that somehow, SAP is not sending the XML data in UTF-8 encoding because the receiving system seems to have a problem with the encoding of the file we are sending.

I am pasting below the two different coding approaches that we have tried. I also have a Word document which shows screen prints of the file we are sending and the error message response we are getting back, if anyone is interested in seeing that documentation.

We need help in knowing if there is some kind of setting somewhere that may be altering the encoding of the file, or if our ABAP logic is incorrect, etc.

Any help is much appreciated!!!

Shannon

• BEGIN CODE EXAMPLE USING STRING ********

• Note: P_XML_OUT is a string of XML, which begins with the header:

• <?xml version="1.0" encoding="UTF-8"?>

rlength = STRLEN( p_xml_out ).

MOVE rlength TO txlen.

CALL METHOD cl_http_client=>create

EXPORTING

host = p_host

• service = '443'

scheme = '2'

• proxy_host = wf_proxy

• proxy_service = wf_port

IMPORTING

client = http_client

EXCEPTIONS

argument_not_found = 1

plugin_not_active = 2

internal_error = 3

OTHERS = 4.

IF sy-subrc <> 0.

• (Error handling logic goes here)

EXIT.

ENDIF.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = '~request_method'

value = 'POST'.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = '~server_protocol'

value = 'HTTP/1.1'.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = '~request_uri'

value = '/integration/HRMS/HRMS.cfc'.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = 'Content-Type'

value = 'text/xml; charset=utf-8'.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = 'Content-Length'

value = txlen.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = 'SOAPAction'

value = ''.

• Set the STRING data

CALL METHOD http_client->request->set_cdata

EXPORTING

data = p_xml_out

offset = 0

length = rlength.

• Send the XML data

CALL METHOD http_client->send

EXCEPTIONS

http_communication_failure = 1

http_invalid_state = 2

http_processing_failed = 3

http_invalid_timeout = 4

OTHERS = 5.

IF sy-subrc <> 0.

• (Error handling goes here)

EXIT.

ENDIF.

• Here we receive the response back from the external system

CALL METHOD http_client->receive

EXCEPTIONS

http_communication_failure = 1

http_invalid_state = 2

http_processing_failed = 3

OTHERS = 4.

IF sy-subrc <> 0.

• (Error handling goes here)

EXIT.

ENDIF.

• Get the returned string of XML data from Recruitmax (the external

• system)

CLEAR wf_string1.

wf_string1 = http_client->response->get_cdata( ).

CALL METHOD http_client->close

EXCEPTIONS

http_invalid_state = 1

OTHERS = 2.

• At this point, we parse the XML that was received back

• from the external system to find any success message

• or error message... This is where we realize that they

• have sent back an error of 'org.xml.sax.SAXParseException: Next

• character must be ">" terminating element "soapenv:Envelope".'

• END CODE EXAMPLE USING STRING ********

This is the change we made in the code to try to use the 2nd approach of using an XSTRING instead of a STRING:

• BEGIN CODE EXAMPLE USING XSTRING INSTEAD *******

.

.

.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = 'SOAPAction'

value = ''.

• Begin Trying to convert string to UTF-8 and send XSTRING ****

DATA: utf8_xml_out TYPE xstring.

DATA: cvto_utf8 TYPE REF TO cl_abap_conv_out_ce.

cvto_utf8 = cl_abap_conv_out_ce=>create( encoding = 'UTF-8' ).

cvto_utf8->write( data = p_xml_out ).

utf8_xml_out = cvto_utf8->get_buffer( ).

rlength = XSTRLEN( utf8_xml_out ).

CALL METHOD http_client->request->set_data

EXPORTING

data = utf8_xml_out

offset = 0

length = rlength.

• End Trying to convert string to UTF-8 and send XSTRING ****

CALL METHOD http_client->send

EXCEPTIONS

http_communication_failure = 1

http_invalid_state = 2

http_processing_failed = 3

http_invalid_timeout = 4

OTHERS = 5.

.

.

.

• BEGIN CODE EXAMPLE USING XSTRING INSTEAD *******

[athavanraja · Accepted solution]

since you are building the xml by hand, you have to use the escape characters for special characters, as xml cannot handle special characters.

for example if an element value is <b>&</b> this should be replaced with <b>&amp;</b>

so before building the xml file , convert the special characters and then build the xml into the variable p_xml_out.

cl_http_utility=>escape_html method can be used for this purpose.

Try this and let us know how it goes.

Regards

Raja

[Former Member · Accepted solution]

Raja,

Thanks for your suggestion. I tried using method cl_http_utility=>escape_html, but it did not change the character é to anything else. It left the character é in the p_xml_out string, so it did not solve the problem.

Is there some other way to convert this type of accented character to something else that would be acceptable?

Thanks,

Shannon

[athavanraja · Accepted solution]

instead of escape_html can you try escape_url method to see if it helps

Regards

Raja

Sorry for the delay, i was on vacation

[Former Member]

Hi,

there is new function in ABAP for escaping characters in different format.

Peter

[Former Member]

We finally found the issue ourselves. The code to convert the String data to an XSTRING with UTF-8 encoding is working correctly:

DATA: utf8_xml_out TYPE xstring.

DATA: cvto_utf8 TYPE REF TO cl_abap_conv_out_ce.

cvto_utf8 = cl_abap_conv_out_ce=>create( encoding = 'UTF-8' ).

cvto_utf8->write( data = p_xml_out ).

utf8_xml_out = cvto_utf8->get_buffer( ).

However, our issue was that we were not finding the length of the new XSTRING and using it when setting header field Content-Length. Therefore, we were accidentally using the length of the original string variable when setting the value of Content-Length. This was causing the last few characters of the file to be truncated because the length of the XSTRING is longer than the length of the String. We made this code correction:

rlength = XSTRLEN( utf8_xml_out ).

MOVE rlength TO txlen.

.

.

.

CALL METHOD http_client->request->set_header_field

EXPORTING

name = 'Content-Length'

value = txlen.

This solved our issue, and now the file is correctly sent in UTF-8 encoding and with the correct length.