Skip to Content
0

sap pi : Not accepting encoding in right manner for turkish language

Oct 06, 2017 at 07:44 PM

104

avatar image
Dear Experts,
I am using an AS2 based scenario,where i am sending a delimited file to partner X.This delimited file contains lots of columns.I am able to produce delimited file well ,and even send to my partner well.
Issue comes when language is turkish.I am informed from partner side that they use encoding UTF-8.But when i specify UTF-8 encoding in my code ,i see some different junk characters.When i use ISO-8859-9,situation is little better but not perfect.
Example: ARŞİV RA  has two special characters "Şİ" which i am failing to get in my final AS2 output by using listed code.On use of ISO-8859-9,i get though characters "ÞÝ" in place of "Şİ".I understood the mystery when i went to this link.
Link of explanation
https://en.wikipedia.org/wiki/ISO/IEC_8859-9
This link explains that Þ = Ş , Ý = İ in ISO-8859-9.
But i want proper characters Şİ .I even tried doing one by one assignment ,by replacing characters reading in byte array.I was able to replace Þ by readable characters only.
But when i substituted same with Ş ,i simply got a 
AR?İV RA indicating a question mark.

Otherwise generally these characters ÞÝ ,i am able to see correctly on notepad by using any turkish windows font.
However ,my partner is not approving the file.How to show these special turkish characters.




Code Listing:


	int flg;
	String complete;


public void transform( TransformationInput transformationInput ,TransformationOutput transformationOutput )
throws StreamTransformationException
{
	


try
{
//*******
	InputStream is= transformationInput.getInputPayload().getInputStream();
    Charset iso88599charset = Charset.forName("ISO-8859-9");
	// Instantiating output stream to write at Target message
	OutputStream os = transformationOutput.getOutputPayload().getOutputStream();
	// building a new document to parse input stream i.e. our source message
	DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
	DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
	Document doc = dBuilder.parse(transformationInput.getInputPayload().getInputStream());
	//Instantiating HeaderParser class
	HeaderParser1 hparser1= new HeaderParser1();
	//retrieving header line of flat file
	String strHeader1=hparser1.parseHeaderString1(doc);
	Map mapParameters = (Map)transformationInput.getInputHeader().getAll();
	mapParameters.put(DynamicConfigurationKey.create("http://sap.com/xi/XI/Dynamic",
	StreamTransformationConstants.DYNAMIC_CONFIGURATION), "");
	DynamicConfiguration conf = (DynamicConfiguration) mapParameters.get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
	//DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File", "FileName");
	DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/AS2/AS2", "AS2Filename");
	//Depending on your requirement edit this logic. Here, NewDynamicName + CurrentDate will be output file name.
	DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd_HHmmss");
	conf.put(key,strHeader1);
//
	


	flg = 1;
	complete = "";
	InputStream inputstream = transformationInput.getInputPayload().getInputStream();
	OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();
    byte[] b = new byte[inputstream.available()]; 
    inputstream.read(b); 
    String inputXML = new String(b);
    Document inDoc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new ByteArrayInputStream(inputXML.getBytes(iso88599charset ))));
    StringWriter writer = new StringWriter();
    doSomething (inDoc.getDocumentElement());
	outputstream.write(new String(complete).getBytes(iso88599charset ));
    
}
catch (Exception exception) {
	getTrace().addDebugMessage(exception.getMessage());
	throw new StreamTransformationException(exception.toString());
}
}
public void doSomething (Node node) {


	String nodeName = node.getNodeName();
	String nodeContent = new String ( node.getTextContent());


	if ( (flg == 1) && ( nodeName == "Details" ) ) {
		flg = 0;
	}


	if ( ( ( nodeName == "Details")  ) && ( flg == 0) ) {
		complete = complete + "\n" ;
		}
	if ( ( nodeName != "Details" ) && ( nodeName != "ns0:MT_CitiBank_PaymentDetails_TR" ) && ( nodeName != "Filename" )  ) {
		complete = complete + nodeContent + "#";
	}
	//System.out.println(node.getNodeName());	
	//System.out.println(node.getTextContent());
	NodeList nodeList = node.getChildNodes();
	for (int i=0;i<nodeList.getLength();i++) {
		Node currentNode = nodeList.item(i);
		if (currentNode.getNodeType() == Node.ELEMENT_NODE ) {
			doSomething(currentNode);
		}
	}
	}














 class HeaderParser1 {    


    public String parseHeaderString1(Document doc){


             


                String strInput="";


               NodeList nList = doc.getElementsByTagName("Details");


                           Node nNode = nList.item(0);


                          Element element= (Element) nNode;


                   //Forming header line 


                      strInput= element.getElementsByTagName("Filename").item(0).getTextContent();


                           return strInput;


                         }


  }

10 |10000 characters needed characters left characters exceeded

I would make things more simple.I have no option to create a delimited file in AS2 reciever channel like in FTP we have.I am forced to use AS2 channel because it is partner requirement and i am not going to send any file such as EDIFACT,ANSI but a flat text delimited file.I have no option than to use Java Function.Beside content transformation to delimited ,i have to support encoding within my java code only.

Now in channel if i give any transform bean even,it is overridden by mapping java function,so no way there too.

Now my java code,how can i correct to support turkish characters .Closest i can make them is look icelandic characters.

0
* Please Login or Register to Answer, Follow or Comment.

5 Answers

Aditya Sharma Oct 08, 2017 at 06:31 AM
0

I want to reemphasize an important fact.

Kindly see if you visit this link

https://en.wikipedia.org/wiki/ISO/IEC_8859-9

This link clearly shows that ISO-8859-9 is created by replacing icelandic chars to turkish.

My problem is at moment if i use getBytes ("ISO-8859-9") through my java function,i get combinations not for turkey but for icelandic.

Ideally i must get for turkey.Why is it so ?

Can this note solve my issue ?

1493845 - AS-JAVA: Problems with the Turkish character setVer

Position	0xD0	0xDD	0xDE	0xF0	0xFD	0xFE
8859-9	Ğ	İ	Ş	ğ	ı	ş
8859-1	Ð	Ý	Þ	ð	ý	þ

Share
10 |10000 characters needed characters left characters exceeded
Aditya Sharma Oct 08, 2017 at 04:06 PM
0

I was hoping that using Locale would help but applying this modified code also has no impact.Show me icelandic chars only despite using ISO-8859-9.




	int flg;
	String complete;
   String cha;
String newstr;
String newinputXML;
char ch[];
char mh;
public void transform( TransformationInput transformationInput ,TransformationOutput transformationOutput )
throws StreamTransformationException
{
	


try
{
//*******
	InputStream is= transformationInput.getInputPayload().getInputStream();
    Charset iso88599charset = Charset.forName("ISO-8859-9");
    Locale trlocale= new Locale("tr", "TR");
    Locale.setDefault (trlocale) ;


	// Instantiating output stream to write at Target message
	OutputStream os = transformationOutput.getOutputPayload().getOutputStream();
	// building a new document to parse input stream i.e. our source message
	DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
	DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
	Document doc = dBuilder.parse(transformationInput.getInputPayload().getInputStream());
	//Instantiating HeaderParser class
	HeaderParser1 hparser1= new HeaderParser1();
	//retrieving header line of flat file
	String strHeader1=hparser1.parseHeaderString1(doc);
	Map mapParameters = (Map)transformationInput.getInputHeader().getAll();
	mapParameters.put(DynamicConfigurationKey.create("http://sap.com/xi/XI/Dynamic",
	StreamTransformationConstants.DYNAMIC_CONFIGURATION), "");
	DynamicConfiguration conf = (DynamicConfiguration) mapParameters.get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
	//DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File", "FileName");
	DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/AS2/AS2", "AS2Filename");
	//Depending on your requirement edit this logic. Here, NewDynamicName + CurrentDate will be output file name.
	DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd_HHmmss");
	conf.put(key,strHeader1);
//
	
flg = 1;
	complete = "";
	InputStream inputstream = transformationInput.getInputPayload().getInputStream();
	OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();
    byte[] b = new byte[inputstream.available()]; 
    inputstream.read(b); 
    String inputXML = new String(b);
// aditya code
cha = "";
newstr = "";
newinputXML = "";
ch = inputXML.toCharArray();
for (int i=0;i<=ch.length-1;i++) {
mh = (char) ch[i];
cha = String.valueOf(mh);
boolean hasUpperCase = !(cha.equals(cha.toLowerCase()));
boolean hasLowerCase = !(cha.equals(cha.toUpperCase()));
if (hasUpperCase == true ) {
newinputXML = newinputXML + cha.toUpperCase(trlocale);
}
if (hasLowerCase == true ) {
newinputXML = newinputXML + cha.toLowerCase(trlocale);
}


if ((hasUpperCase == false ) && (hasLowerCase == false)) {
newinputXML = newinputXML + cha;
}
}


// finished
    Document inDoc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new ByteArrayInputStream(newinputXML.getBytes("ISO-8859-9"))));
    StringWriter writer = new StringWriter();
    doSomething (inDoc.getDocumentElement());


// aditya code
cha = "";
newstr = "";


ch = complete.toCharArray();
for (int i=0;i<=ch.length-1;i++) {
mh = (char) ch[i];
cha = String.valueOf(mh);
boolean hasUpperCase = !(cha.equals(cha.toLowerCase()));
boolean hasLowerCase = !(cha.equals(cha.toUpperCase()));
if (hasUpperCase == true ) {
newstr = newstr + cha.toUpperCase(trlocale);
}
if (hasLowerCase == true ) {
newstr = newstr + cha.toLowerCase(trlocale);
}


if ((hasUpperCase == false ) && (hasLowerCase == false)) {
newstr = newstr + cha;
}
}


// finished
complete = newstr;
	outputstream.write(new String(newstr).getBytes("ISO-8859-9"));
}
catch (Exception exception) {
	getTrace().addDebugMessage(exception.getMessage());
	throw new StreamTransformationException(exception.toString());
}
}
public void doSomething (Node node) {


	String nodeName = node.getNodeName();
	String nodeContent = new String ( node.getTextContent());


	if ( (flg == 1) && ( nodeName == "Details" ) ) {
		flg = 0;
	}


	if ( ( ( nodeName == "Details")  ) && ( flg == 0) ) {
		complete = complete + "\n" ;
		}
	if ( ( nodeName != "Details" ) && ( nodeName != "ns0:MT_CitiBank_PaymentDetails_TR" ) && ( nodeName != "Filename" )  ) {
		complete = complete + nodeContent + "#";
	}
	//System.out.println(node.getNodeName());	
	//System.out.println(node.getTextContent());
	NodeList nodeList = node.getChildNodes();
	for (int i=0;i<nodeList.getLength();i++) {
		Node currentNode = nodeList.item(i);
		if (currentNode.getNodeType() == Node.ELEMENT_NODE ) {
			doSomething(currentNode);
		}
	}
	}














 class HeaderParser1 {    


    public String parseHeaderString1(Document doc){


             


                String strInput="";


               NodeList nList = doc.getElementsByTagName("Details");


                           Node nNode = nList.item(0);


                          Element element= (Element) nNode;


                   //Forming header line 


                      strInput= element.getElementsByTagName("Filename").item(0).getTextContent();


                           return strInput;


                         }


  }
Share
10 |10000 characters needed characters left characters exceeded
Aditya Sharma Oct 07, 2017 at 12:31 PM
0

Does seem sap community site is dead now.Really it has turned from best to now a sinking ship.

Share
10 |10000 characters needed characters left characters exceeded
Manoj K Oct 07, 2017 at 01:03 PM
0

Hi Aditya,

Have you tried MTB/ TextCodePageConversion Bean as the second last module in your reciever channel ?

Br,

Manoj

Show 5 Share
10 |10000 characters needed characters left characters exceeded

AF_Transform bean I cant try.I failed to understand the analogy behind the development of this bean.

I have 15000000 fields.How can I give each and every field?

Now coming to Anonymizer,can u explain which namespace Mr Bean will approve ?

0

AFAIK fieldname parameter is not mandatory with MTB when converting from XML to plain.

Why are using Anonymizer bean , it is used to convert the xml encoding . i don't that that will be applicable in your case.

0

My content is not xml.I am actually passing text/plain content via AS2 reciever.

I transformed by parsing xml in this mentioned udf to a delimited format.Channel recieves plain text content.

If i pass to FTP and give encoding ,it works.But i have to use AS2 receiver channel and thats a constraint.

Yes, Any java module to transform content of text type ?.

I have assumed that SAP has not given any encoding for turkish language.The one declared as ISO-8559-9 has now turkish letters replaced by icelandic ones.Talking independently about Turkish ,UTF-8 also has no mention of turkish use.I have even tried on rich editor to substitute turkish symbols with icelandic as only six letters need to be worked out.

They get substituted but come output as ?.

0

Also i figured out that i am facing issue because sap channel libraries are more upto date compare to jdk1.6 old java API library .So channels encoding work well.

0

It does seem i am little close.How can i set locale as TR in the above code.I tried but doesnot work.

Locale trlocale= new Locale("tr", "TR");
    Locale.setDefault (trlocale) ;
0
Aditya Sharma Oct 09, 2017 at 12:52 PM
0

No help !

Share
10 |10000 characters needed characters left characters exceeded