HCI - Adding Byte Order Mark (BOM) to UTF-8 file content

[Former Member]

Dear Experts,

I'm trying to add BOM (Byte Order Mark) characters to UTF-8 file content thru Groovy script while sending file to 3rd party SFTP server.

• Added CamelCharsetName with UTF-8 as Header parameter

• In Groovy script, tried appending BOM with payload content

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;

def Message processData(Message message) {
            
        Map headerMap = message.getHeaders();
        def payload = message.getBody(java.lang.String);
        String bom = new String("\u00EF\u00BB\u00BF"); //"0xEF,0xBB,0xBF";
                
        //Add ByteOrderMark to payload
        payload = bom+payload;
        message.setBody(payload);
    
return message;
}

The file created at the target does not have correct BOM characters and failing in target. When examined the target file with Hex editor we can clearly see that the BOM is not correct (the same code in Groovy console is creating correct BOM so editors like Notepad & Notepad++ recognize correctly as UTF-8 with BOM).

Correct BOM & Text Content (expected) as viewed in -- HEX -- Mode

Correct BOM & Text Content (expected) as viewed in -- Normal -- Mode

Incorrect BOM (produced by HCI thru Groovy) in -- HEX -- Mode

Anybody experienced similar issue and found any work around to fix the issue?

Thanks in advance.

Ananth Chinnaraj

[former_member75481]

Hello,

and if i want to remove BOM from the file when the input file is UTF8BOM? How would the script look like?

[Former Member · Accepted solution]

Hi All,

After a lot of trial and error, the following code worked for me.

def Message processData(Message message) {
            
        Map headerMap = message.getHeaders();
        def payload = message.getBody(java.lang.String) as String;
       
        String charset = "utf-8";
        byte[] BOM = [0xEF,0xBB,0xBF];
        
        String byteordermark = new String (BOM, charset); 
        String s1 = new String(payload.decodeBase64(),charset); // payload is encoded in Base64 format
        String s2 = new String(BOM,charset) + s1;
        
        message.setBody(s2);

}

Thanks,

Ananth

[psethi · Accepted solution]

Hi Ananth,

I am intgerating SuccessFactors with Concur and they want UTF-8 with BOM. I have tried scripts that you mentioned in the post along with some changes as well but nothing really works. Do you know if it's still possible to enforce BOM?

Concur needs a text file so I am generating that and then set CamelCharsetName as UTF-8 before the script.

I am using below script :

import com.sap.it.api.mapping.*;

import com.sap.gateway.ip.core.customdev.util.Message;

import java.util.HashMap;

import java.nio.charset.StandardCharsets;

def Message processData(Message message) {

Map headerMap = message.getHeaders();

def payload = message.getBody(java.lang.String) as String;

byte[] bom = [0xEF,0xBB,0xBF];

byte[] document = payload.getBytes(StandardCharsets.UTF_8);

byte[] withBom = new byte[bom.length + document.length];

System.arraycopy(bom,0,withBom,0,bom.length);

System.arraycopy(document,0,withBom,bom.length,document.length);

String s2 = new String(withBom); message.setBody(s2);

return message;

}

[former_member649396 · Accepted solution]

Hi Ananth,

The script is not working.

Thanks

Rajat