Handling Special Characters

[Former Member]

Dear Friends,

In our interface we need to handle special characters which are coming from the source. I mean when there are any special characters from Source it should not fail in PI.

Is there any ways to handle these special characters ?

Currently lots of messages from source are coming with special charcters due to this we are encountering lots of failures in PI.

Kindly guide me.

Renu

[RaghuVamseedhar]

Hi Renu,

I think, messages are failing because they contain XML reserved characters. i.e., XML Messages which are failing are not well formed XML messages.

Note: - quote ("), apostrophe ('), ampersand (&), less than (<), greater than (>) are special characters in XML. They should not be present in data. http://www.w3.org/TR/REC-xml/

Not Well formed XML:

<Name>Raghuu2019s</Name>   <Name> Raghu & Vamsee </Name>

Well formed XML:

<Name>Raghu&apos;s</Name>   <Name>Raghu &amp; Vamsee</Name>

First take the payload which has failed, take out (delete) reserved characters from data.

< chatlog >6. If you u2019 re still having problems < chatlog > <chatlog> Raghu & Vamse </chatlog>

Delete ' " &, if they are present in data

Now test with new payload (without reserved characters) on your old interface (mapping before you have raised this thread in SDN).

If, messages are processed successfully. Then, issue is with reserved characters, we need to write java mapping to replace reserved characters with escape sequence. I feel there is no need to for character encoding change using XSLT . Else issue is with something else.

Please share test results and error message in your production system (before you have raised this thread in SDN).

Regards,

Raghu_Vamsee

[baskar_gopalakrishnan2]

This link might be helpful. I used it before...

http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/9420 [original link is broken] [original link is broken] [original link is broken]

[Former Member]

Hi Renu,

Ref: http://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79f...

Thanks,

[Former Member]

Hi Renu,

There are various ways of handling special characters based on the requirement we need to take decisions:

1. You can use CDATA section of xml to avoid parsing of the special characters in ur xml, for this u can use Java Mapping or XSLT mapping based on your problem.

2. you can use encoding techniques based of different adapters.

So it would be good that you describe your requirement in some details for others to help you. In the mean time you can also refer the below running thread:

[;

[stefan_grube]

Can you describe, where the data come from, which adapter is used and which characters do you mean?

[Former Member]

Dear Friends,

We are getting message from Third party Vendor. It is HTTP post from Vendor.

In the message there is a field called Chatlog. we will be getting any kind of special characters.

We will get these characters from many languages. ( like protugese , Spanish , French , detuch ,chinese etc...)

When any type of special characters are coming from the Vendor. We need to handle this special characters.

Kindly let me know if you need any more information.

'

KIndly guide me.

Renu

[stefan_grube]

> Kindly let me know if you need any more information.

A lot.

Is the incoming message an XML message or plain text, do you know the encoding of the message, is it always the same or could it be different, what do you want to do with the message.

Please explain your scenario in detail.

In principle there is nothing special with chinese, french, spanish characters, they are treated as any other character.

The incoming encoding should be UTF-8, if not you can change the encoding.

Habe a look in this how-to guide:

http://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/502991a2-45d9-2910-d99f-8aba5d79f...

[Former Member]

Hi Stefan,

Scenario:

Vendor -

---

>PI -

---

> Siebel

Vendor to PI is HTTP post

1. Incoming message is : xml format

2. Encoding Format is : UTF-8 (Always)

what do you want to do with the message ?

When any special character is coming from source message from a particular fieldd called <Chatlog> we need to pass the message to target.

Due to this special character we are getting so many failures and eventually leads to loss of critical business transactions.

We need to stop these failures from PI standpoint.

Can this xslt code will resolve our issue ?

<xsl:stylesheet version="1.0"

xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<xsl:output method="xml" encoding="ISO-8859-1"/>

<xsl:template match="/">

<xsl:copy-of select="*" />

</xsl:template>

</xsl:stylesheet> {code)

[stefan_grube]

what is the target and what are the failures?

You cannot just remove charaters from a message, that would corrupt the message!

[Former Member]

I mean failures are due to special characters in the chatlog field.

The targer system is Siebel.

How can we resolve this issue ?

[stefan_grube]

Sorry, I cannot help you.

It seems that you are not able to give any useful information.

[Former Member]

Hi Stefan

Sample source message

<?xml version="1.0" encoding="UTF-8" ?>
<session>
	<sessionid>55555555</sessionid>
	<techid>1111111</techid>
	<techssoid></techssoid>
	<techname>dummy</techname>
	<tech>dummy</tech>
	<techdescr></techdescr>
	<cfield0>sam rogen</cfield0>
      <cfield2>dummy/rint</cfield2>
	<cfield3>Other|NA|en</cfield3>
	<cfield4>dummy</cfield4>
	<cfield5>4556395000</cfield5>
	<tracking0>LiveChatform</tracking0>
	<chatlog>6:17 AM Session  has been started by the system...

6:32 AM Medwin: Sure let me check the settings for you.
6:34 AM Medwin: The  type is Imap.
6:35 AM Ron Goldstein: ok
6:35 AM Medwin:  type: IMAP   Incoming server:  imap.aim.com   Username: &lt;username before  @&gt;   Port: 143   Encryption: None
6:35 AM Medwin: Outgoing server:  smtp.aim.com   Use Authentication: On   Username: &lt;username before  @&gt;   Port: 587   Encryption: None.
6. If youu2019re still having problems with the same issue, contact our Chat Support or Voice Support and refer to Chat Session ID # 77419452 to continue working on this issue. Thank you for contacting . Our chat session is going to disconnect now.
7:30 AM The technician ended the session.
</chatlog>
	<notes></notes>
	<waitingtime>55</waitingtime>
	<pickuptime>2011-01-03 00:46:58</pickuptime>
	<closingtime>2011-01-03 02:00:45</closingtime>
	<worktime>1592</worktime>
	<lastactiontime>2011-01-03 02:00:24</lastactiontime>
	<transmitted>75528</transmitted>
	<platform>Windows 7</platform>
	<tsurvey0>Using </tsurvey0>
	<tsurvey3>Unresolved - incomplete TS</tsurvey3>
	<tsurvey4>cannot send </tsurvey4>
</session>

[Former Member]

Hi Stefan,

I have sent the sample source messsage which is coming form Vendor via http post.

we are not using any sender adapter.

When it is coming into XI pipeline it is failing due to special character in the source message.

You can open this source message and you can see the rootcause.

Our requiremnt is : How to avoid failures due to such kind of special characters?

I hope i am able to give the required information to you .

Renu.

[Former Member]

Hi Renu,

There is no issue with the encoding here. You just need to add your chatlog field with in a CDATA section. To add CDATA you can either go with xslt or java mapping. Also you can try to concatenate the String

<[!CDATA[ (as a constant) to <CHATLOG to field>  (as constant) ]]>

.

For Java mapping and xslt you can google the codes. I will also try to provide you the code in some time.

Regards,

Rahul Grover

Edited by: rahul grover on Jan 5, 2011 9:52 AM

[Former Member]

Ok Rahul. I will wait for your input.

[Former Member]

Hi renu,

Try the following xslt mapping.

<?xml version="1.0" encoding="ISO-8859-1"?>
<!-- Edited by XMLSpyu00AE -->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match = "/">
<sessionid>
  <xsl:value-of select="session/sessionid"/>
</sessionid>
<techid>
  <xsl:value-of select="session/techid"/>
</techid>
extend this for all the other fields of your xml 
.
.
<chatlog>
<xsl:text disable-output-escaping="yes"><![CDATA[<![CDATA[

]]
>


 
]]>

Hope this will help you.

Regards,

rahul

[Former Member]

Hi Rahul,

Do i need to place this xslt mapping before message mapping step in Interface Mapping step.

In Interface mapping

Source -

---

> 1. Xslt mapping -

---

> Target

2.Message mapping

Kindly suggest me.

[Former Member]

Do i need to place this xslt mapping before message mapping step in Interface Mapping step.

Yes !!

[Former Member]

Hi Rahul,

When i am executing . I am getting following error as follows :

SAP:Category>Application</SAP:Category>

<SAP:Code area="MAPPING">EXCEPTION_DURING_EXECUTE</SAP:Code>

<SAP:P1>com/sap/xi/tf/_MM_LMIChat_</SAP:P1>

<SAP:P2>com.sap.aii.utilxi.misc.api.BaseRuntimeException:</SAP:P2>

<SAP:P3>Fatal Error: com.sap.engine.lib.xml.parser.ParserE</SAP:P3>

<SAP:P4>xception: XMLParser: No data allowed here (:main:~</SAP:P4>

<SAP:AdditionalText />

<SAP:Stack>Runtime exception occurred during application mapping com/sap/xi/tf/_MM_Chat_; com.sap.aii.utilxi.misc.api.BaseRuntimeException:Fatal Error: com.sap.engine.lib.xml.parser.ParserException: XMLParser: No data allowed here (:main:~</SAP:Stack>

<SAP:Retry>M</SAP:Retry>

</SAP:Error>

[Former Member]

The xslt code which i have placed in Interface mapping step :

<?xml version="1.0" encoding="ISO-8859-1"?>
<!-- Edited by XMLSpyu00AE -->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 xmlns:map="java:java.util.Map"
  xmlns:dyn="java:com.sap.aii.mapping.api.DynamicConfiguration"
  xmlns:key="java:com.sap.aii.mapping.api.DynamicConfigurationKey"
   xmlns:ns0="http://sample.com/Chat">
<xsl:output method="xml" encoding="UTF-8" indent="yes"/>
<xsl:template match = "/">
<sessionid>
  <xsl:value-of select="session/sessionid"/>
</sessionid>
<techid>
  <xsl:value-of select="session/techid"/>
</techid>
<techssoid> 
<xsl:value-of select="session/techssoid"/>
</techssoid>
<techname>
<xsl:value-of select="session/techname"/>
</techname>

<techemail>

<xsl:value-of select="session/techemail"/>
</techemail>
<techdescr> 
<xsl:value-of select="session/techdescr"/>
</techdescr> 
<cfield0>
<xsl:value-of select="session/cfield0"/>
</cfield0>
<cfield1>
<xsl:value-of select="session/cfield1"/>
</cfield1>

<cfield2>
<xsl:value-of select="session/cfield2"/>
</cfield2>
<cfield3>
<xsl:value-of select="session/cfield3"/>
</cfield3> 
<cfield4>
<xsl:value-of select="session/cfield4"/>
</cfield4>
<cfield5>
<xsl:value-of select="session/cfield5"/>
</cfield5>
<tracking0> 
<xsl:value-of select="session/tracking0"/>
</tracking0> 
<chatlog>
<xsl:text disable-output-escaping="yes"><![CDATA[<![CDATA[

]]
>

 

 















 

 
















 
 




]]>

[Former Member]

<?xml version="1.0" encoding="ISO-8859-1"?>
<!-- Edited by XMLSpy® -->
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
 xmlns:map="java:java.util.Map"
  xmlns:dyn="java:com.sap.aii.mapping.api.DynamicConfiguration"
  xmlns:key="java:com.sap.aii.mapping.api.DynamicConfigurationKey"
   xmlns:ns0="http://sample.com/Chat">
<xsl:output method="xml" encoding="UTF-8" indent="yes"></xsl:output>

remove the extra ">" in third line, it should be

 <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
xmlns:map="java:java.util.Map"
  xmlns:dyn="java:com.sap.aii.mapping.api.DynamicConfiguration"
  xmlns:key="java:com.sap.aii.mapping.api.DynamicConfigurationKey"
   xmlns:ns0="http://sample.com/Chat">
<xsl:output method="xml" encoding="UTF-8" indent="yes"></xsl:output>

Regards,

Rahul

[Former Member]

Hi Rahul,

I made the changes as you suggested. I am getting the following error :

Runtime Exception when executing application mapping program com/sap/xi/tf/_MM_LMIChat_; Details: com.sap.aii.utilxi.misc.api.BaseRuntimeException; Fatal Error: com.sap.engine.lib.xml.parser.ParserException: XMLParser: No data allowed here (:main:, row:3, col:2)

Please guide me.

[Former Member]

Hi Renu,

Please check the inbound (sender) payload in sxmb_moni. Is the sender XML valid???

You can test your xslt code here:

[http://www.w3schools.com/xsl/tryxslt.asp?xmlfile=cdcatalog&xsltfile=cdcatalog]

Regards

Rahul

Edited by: rahul grover on Jan 6, 2011 12:46 PM

[Former Member]

HI Rahul,

The inbound message is vaild. I tried to post one successful message in Dev. At that juncture only i got this error:

Can this xslt mapping work when the Inbound message is having xml parsing error due to special character ?

since this is my exact requirement.----> When the inbound message is not valid due to chatlog field we need to add CDATA

Is this Possible since vendor sending message via http post ?

[Former Member]

Hi Renu,

Without a valid xml @ inbound it is not possible to work with this xslt mapping or any ohter graphical mapping. If it is a matter to escape the encoding of the special characters with in PI we can use this xslt mapping.

In your case it is better to ask a valid xml from client. Or atleast tell them to send the chatlog field with in CDATA section.

Regards

Rahul