Skip to Content

sap pi : Not accepting encoding in right manner for turkish language

Dear Experts,
I am using an AS2 based scenario,where i am sending a delimited file to partner X.This delimited file contains lots of columns.I am able to produce delimited file well ,and even send to my partner well.
Issue comes when language is turkish.I am informed from partner side that they use encoding UTF-8.But when i specify UTF-8 encoding in my code ,i see some different junk characters.When i use ISO-8859-9,situation is little better but not perfect.
Example: ARŞİV RA  has two special characters "Şİ" which i am failing to get in my final AS2 output by using listed code.On use of ISO-8859-9,i get though characters "ÞÝ" in place of "Şİ".I understood the mystery when i went to this link.
Link of explanation
https://en.wikipedia.org/wiki/ISO/IEC_8859-9
This link explains that Þ = Ş , Ý = İ in ISO-8859-9.
But i want proper characters Şİ .I even tried doing one by one assignment ,by replacing characters reading in byte array.I was able to replace Þ by readable characters only.
But when i substituted same with Ş ,i simply got a 
AR?İV RA indicating a question mark.

Otherwise generally these characters ÞÝ ,i am able to see correctly on notepad by using any turkish windows font.
However ,my partner is not approving the file.How to show these special turkish characters.




Code Listing:


	int flg;
	String complete;


public void transform( TransformationInput transformationInput ,TransformationOutput transformationOutput )
throws StreamTransformationException
{
	


try
{
//*******
	InputStream is= transformationInput.getInputPayload().getInputStream();
    Charset iso88599charset = Charset.forName("ISO-8859-9");
	// Instantiating output stream to write at Target message
	OutputStream os = transformationOutput.getOutputPayload().getOutputStream();
	// building a new document to parse input stream i.e. our source message
	DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
	DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
	Document doc = dBuilder.parse(transformationInput.getInputPayload().getInputStream());
	//Instantiating HeaderParser class
	HeaderParser1 hparser1= new HeaderParser1();
	//retrieving header line of flat file
	String strHeader1=hparser1.parseHeaderString1(doc);
	Map mapParameters = (Map)transformationInput.getInputHeader().getAll();
	mapParameters.put(DynamicConfigurationKey.create("http://sap.com/xi/XI/Dynamic",
	StreamTransformationConstants.DYNAMIC_CONFIGURATION), "");
	DynamicConfiguration conf = (DynamicConfiguration) mapParameters.get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
	//DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File", "FileName");
	DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/AS2/AS2", "AS2Filename");
	//Depending on your requirement edit this logic. Here, NewDynamicName + CurrentDate will be output file name.
	DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd_HHmmss");
	conf.put(key,strHeader1);
//
	


	flg = 1;
	complete = "";
	InputStream inputstream = transformationInput.getInputPayload().getInputStream();
	OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();
    byte[] b = new byte[inputstream.available()]; 
    inputstream.read(b); 
    String inputXML = new String(b);
    Document inDoc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new ByteArrayInputStream(inputXML.getBytes(iso88599charset ))));
    StringWriter writer = new StringWriter();
    doSomething (inDoc.getDocumentElement());
	outputstream.write(new String(complete).getBytes(iso88599charset ));
    
}
catch (Exception exception) {
	getTrace().addDebugMessage(exception.getMessage());
	throw new StreamTransformationException(exception.toString());
}
}
public void doSomething (Node node) {


	String nodeName = node.getNodeName();
	String nodeContent = new String ( node.getTextContent());


	if ( (flg == 1) && ( nodeName == "Details" ) ) {
		flg = 0;
	}


	if ( ( ( nodeName == "Details")  ) && ( flg == 0) ) {
		complete = complete + "\n" ;
		}
	if ( ( nodeName != "Details" ) && ( nodeName != "ns0:MT_CitiBank_PaymentDetails_TR" ) && ( nodeName != "Filename" )  ) {
		complete = complete + nodeContent + "#";
	}
	//System.out.println(node.getNodeName());	
	//System.out.println(node.getTextContent());
	NodeList nodeList = node.getChildNodes();
	for (int i=0;i<nodeList.getLength();i++) {
		Node currentNode = nodeList.item(i);
		if (currentNode.getNodeType() == Node.ELEMENT_NODE ) {
			doSomething(currentNode);
		}
	}
	}














 class HeaderParser1 {    


    public String parseHeaderString1(Document doc){


             


                String strInput="";


               NodeList nList = doc.getElementsByTagName("Details");


                           Node nNode = nList.item(0);


                          Element element= (Element) nNode;


                   //Forming header line 


                      strInput= element.getElementsByTagName("Filename").item(0).getTextContent();


                           return strInput;


                         }


  }

Add comment
10|10000 characters needed characters exceeded

  • I would make things more simple.I have no option to create a delimited file in AS2 reciever channel like in FTP we have.I am forced to use AS2 channel because it is partner requirement and i am not going to send any file such as EDIFACT,ANSI but a flat text delimited file.I have no option than to use Java Function.Beside content transformation to delimited ,i have to support encoding within my java code only.

    Now in channel if i give any transform bean even,it is overridden by mapping java function,so no way there too.

    Now my java code,how can i correct to support turkish characters .Closest i can make them is look icelandic characters.

  • Get RSS Feed

5 Answers

  • Oct 08, 2017 at 06:31 AM

    I want to reemphasize an important fact.

    Kindly see if you visit this link

    https://en.wikipedia.org/wiki/ISO/IEC_8859-9

    This link clearly shows that ISO-8859-9 is created by replacing icelandic chars to turkish.

    My problem is at moment if i use getBytes ("ISO-8859-9") through my java function,i get combinations not for turkey but for icelandic.

    Ideally i must get for turkey.Why is it so ?

    Can this note solve my issue ?

    1493845 - AS-JAVA: Problems with the Turkish character setVer

    Position	0xD0	0xDD	0xDE	0xF0	0xFD	0xFE
    8859-9	Ğ	İ	Ş	ğ	ı	ş
    8859-1	Ð	Ý	Þ	ð	ý	þ
    
    
    Add comment
    10|10000 characters needed characters exceeded

  • Oct 08, 2017 at 04:06 PM

    I was hoping that using Locale would help but applying this modified code also has no impact.Show me icelandic chars only despite using ISO-8859-9.

    
    
    
    	int flg;
    	String complete;
       String cha;
    String newstr;
    String newinputXML;
    char ch[];
    char mh;
    public void transform( TransformationInput transformationInput ,TransformationOutput transformationOutput )
    throws StreamTransformationException
    {
    	
    
    
    try
    {
    //*******
    	InputStream is= transformationInput.getInputPayload().getInputStream();
        Charset iso88599charset = Charset.forName("ISO-8859-9");
        Locale trlocale= new Locale("tr", "TR");
        Locale.setDefault (trlocale) ;
    
    
    	// Instantiating output stream to write at Target message
    	OutputStream os = transformationOutput.getOutputPayload().getOutputStream();
    	// building a new document to parse input stream i.e. our source message
    	DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
    	DocumentBuilder dBuilder = dbFactory.newDocumentBuilder();
    	Document doc = dBuilder.parse(transformationInput.getInputPayload().getInputStream());
    	//Instantiating HeaderParser class
    	HeaderParser1 hparser1= new HeaderParser1();
    	//retrieving header line of flat file
    	String strHeader1=hparser1.parseHeaderString1(doc);
    	Map mapParameters = (Map)transformationInput.getInputHeader().getAll();
    	mapParameters.put(DynamicConfigurationKey.create("http://sap.com/xi/XI/Dynamic",
    	StreamTransformationConstants.DYNAMIC_CONFIGURATION), "");
    	DynamicConfiguration conf = (DynamicConfiguration) mapParameters.get(StreamTransformationConstants.DYNAMIC_CONFIGURATION);
    	//DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/System/File", "FileName");
    	DynamicConfigurationKey key = DynamicConfigurationKey.create("http://sap.com/xi/XI/AS2/AS2", "AS2Filename");
    	//Depending on your requirement edit this logic. Here, NewDynamicName + CurrentDate will be output file name.
    	DateFormat dateFormat = new SimpleDateFormat("yyyyMMdd_HHmmss");
    	conf.put(key,strHeader1);
    //
    	
    flg = 1;
    	complete = "";
    	InputStream inputstream = transformationInput.getInputPayload().getInputStream();
    	OutputStream outputstream = transformationOutput.getOutputPayload().getOutputStream();
        byte[] b = new byte[inputstream.available()]; 
        inputstream.read(b); 
        String inputXML = new String(b);
    // aditya code
    cha = "";
    newstr = "";
    newinputXML = "";
    ch = inputXML.toCharArray();
    for (int i=0;i<=ch.length-1;i++) {
    mh = (char) ch[i];
    cha = String.valueOf(mh);
    boolean hasUpperCase = !(cha.equals(cha.toLowerCase()));
    boolean hasLowerCase = !(cha.equals(cha.toUpperCase()));
    if (hasUpperCase == true ) {
    newinputXML = newinputXML + cha.toUpperCase(trlocale);
    }
    if (hasLowerCase == true ) {
    newinputXML = newinputXML + cha.toLowerCase(trlocale);
    }
    
    
    if ((hasUpperCase == false ) && (hasLowerCase == false)) {
    newinputXML = newinputXML + cha;
    }
    }
    
    
    // finished
        Document inDoc = DocumentBuilderFactory.newInstance().newDocumentBuilder().parse(new InputSource(new ByteArrayInputStream(newinputXML.getBytes("ISO-8859-9"))));
        StringWriter writer = new StringWriter();
        doSomething (inDoc.getDocumentElement());
    
    
    // aditya code
    cha = "";
    newstr = "";
    
    
    ch = complete.toCharArray();
    for (int i=0;i<=ch.length-1;i++) {
    mh = (char) ch[i];
    cha = String.valueOf(mh);
    boolean hasUpperCase = !(cha.equals(cha.toLowerCase()));
    boolean hasLowerCase = !(cha.equals(cha.toUpperCase()));
    if (hasUpperCase == true ) {
    newstr = newstr + cha.toUpperCase(trlocale);
    }
    if (hasLowerCase == true ) {
    newstr = newstr + cha.toLowerCase(trlocale);
    }
    
    
    if ((hasUpperCase == false ) && (hasLowerCase == false)) {
    newstr = newstr + cha;
    }
    }
    
    
    // finished
    complete = newstr;
    	outputstream.write(new String(newstr).getBytes("ISO-8859-9"));
    }
    catch (Exception exception) {
    	getTrace().addDebugMessage(exception.getMessage());
    	throw new StreamTransformationException(exception.toString());
    }
    }
    public void doSomething (Node node) {
    
    
    	String nodeName = node.getNodeName();
    	String nodeContent = new String ( node.getTextContent());
    
    
    	if ( (flg == 1) && ( nodeName == "Details" ) ) {
    		flg = 0;
    	}
    
    
    	if ( ( ( nodeName == "Details")  ) && ( flg == 0) ) {
    		complete = complete + "\n" ;
    		}
    	if ( ( nodeName != "Details" ) && ( nodeName != "ns0:MT_CitiBank_PaymentDetails_TR" ) && ( nodeName != "Filename" )  ) {
    		complete = complete + nodeContent + "#";
    	}
    	//System.out.println(node.getNodeName());	
    	//System.out.println(node.getTextContent());
    	NodeList nodeList = node.getChildNodes();
    	for (int i=0;i<nodeList.getLength();i++) {
    		Node currentNode = nodeList.item(i);
    		if (currentNode.getNodeType() == Node.ELEMENT_NODE ) {
    			doSomething(currentNode);
    		}
    	}
    	}
    
    
    
    
    
    
    
    
    
    
    
    
    
    
     class HeaderParser1 {    
    
    
        public String parseHeaderString1(Document doc){
    
    
                 
    
    
                    String strInput="";
    
    
                   NodeList nList = doc.getElementsByTagName("Details");
    
    
                               Node nNode = nList.item(0);
    
    
                              Element element= (Element) nNode;
    
    
                       //Forming header line 
    
    
                          strInput= element.getElementsByTagName("Filename").item(0).getTextContent();
    
    
                               return strInput;
    
    
                             }
    
    
      }
    
    Add comment
    10|10000 characters needed characters exceeded

  • Oct 07, 2017 at 12:31 PM

    Does seem sap community site is dead now.Really it has turned from best to now a sinking ship.

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 07, 2017 at 01:03 PM

    Hi Aditya,

    Have you tried MTB/ TextCodePageConversion Bean as the second last module in your reciever channel ?

    Br,

    Manoj

    Add comment
    10|10000 characters needed characters exceeded

  • Oct 09, 2017 at 12:52 PM

    No help !

    Add comment
    10|10000 characters needed characters exceeded