Encoding in Sender/Receiver Channel - Junk Characters

[Former Member]

Hi Experts,

I have an FTP to PI to SFTP scenario. I am getting a file with .dat extension (a flat file ) which the PI sender channel is reading in the binary format, giving it to the receiver channel.

The receiver channel is using SFTP adapter, giving some name to the file and dumping it on some location.

In the incoming file I have the junk following characters coming at the very beginning of the file

This characters are transmitted to the output file as well. Source system is not willing to do any changes to fix this. Hence I was looking to explore some possibility with encodings in PI receiver/sender channels wherein I can get rid of this characters.

On the sender side (FTP connection parameters in the sender channel) I tried

On the Source tab Transfer Mode : Binary

On the Processing tab File Type : Text and File Encoding : UTF-8

but I could not solve the problem.

Any pointers in the right direction will be highly appreciated.

Regards

[Former Member]

Any suggestions SDNers ?

Regards

[Ryan-Crosby]

Hi Upendra,

You could write a simple java mapping to strip those first three characters of the BOM (byte order mark) so it doesn't get written to the output stream.

Regards,

Ryan Crosby

[iaki_vila]

Hi Upendra,

You can also try using an OS command before to take the file, check this thread to skip the junk chars:

http://stackoverflow.com/questions/1068650/using-awk-to-remove-the-byte-order-mark

Regards.

[ambrish_mishra]

Hi Upendra,

Try reading it as Text and UTF-8.

Ambrish

[Former Member]

Cannot do it, they are transmitting the files in .dat format, which the PI sender channel is reading in the binary format.

Regards

[Former Member]

Hello,

Why don't u ask source system to remove these junk characters (which are of no use)?

But having said that if it is a special lang characters which is getting distorted while coming into PI then u can use ISO-8859-1 encoding in the sender channel and then check.

Thanks

Amit Srivastava

[Former Member]

The incoming files are being read as binary files and I cannot ask the source system to change these to text files. So I am not sure how do I use the encoding that you have mentioned above.