File with huge data

[Former Member]

Hi Experts,

I am going to send a file which will be a huge file with 1GB size. What precautions should I take to process the huge file using XI? Scenario is File to IDoc. Suggest me with different options to send a huge file thourgh XI without performance issue.

Regards,

Ramesh.

[Former Member · Accepted solution]

Hi Ramesh,

First suggestion to you would be to try using RMFT.(RepliWeb Managed File Transfer). this is a tool which can be used to do massive data transfer with many file related operations simplified.Try exploring this,it will prove a good option now and in future too for your organization and save XI too!!!....[more info|http://www.repliweb.com/products/rmft/index.php]

The other options would be as suggested in the below mentioned blogs or forums ,have a look

1. [link|http://www.sdn.sap.com/irj/scn/weblogs;jsessionid=(J2EE3414800)ID2129061550DB01842675584212478734End?blog=/pub/wlg/2737]

2.

3.

4.

5.

6.

7. search sdn forums ,there are lots of discussions going on this!

Thanks and regards,

Srini

[rajasekhar_reddy14]

Hi Ramesh,

Use record set per message option in File sender channel, and give value aproximately 250 like that, check the performnce,if its good, then use 250,if you are facing problems then reduce the value.

Cheers,

Raj

[Former Member]

---

Edited by: on Sep 13, 2009 12:57 AM

Edited by:p 13 on Se, 2009 12:57 AM

[agasthuri_doss]

Ramesh,

>Scenario is File to IDoc

Hence Chunk the file and send it.

Cheers

Agasthuri

[Former Member]

Hi,

How do I chunk the file? Could you please elaborate?

Regards,

Ramesh.

[rajasekhar_reddy14]

Hi use append mode in reciever side if you using File Adapter.

[agasthuri_doss]

Hey,

>file which will be a huge file with 1GB size. How do I chunk the file? Could you please elaborate?

Sender side can chunk the file and send or in File adapter use record set where the file will be broken and read automatically

Cheers

Agasthuri

[Former Member]

Hi Ramesh,

You can use File for this, but you might like to do chunking.

Few days back I was working on this huge volume of data going to BW and then being used by Business Objects. I was picking up 16 megs at a time, with read time of 20-25 secs and processing time of 37-42 secs.

Mind you, I was not using any BPM.

So, if you using BPM, you should better chunk it further.

Also, you should check with the Business user, how much time window they can provide PI before they expect the message to be processed at their end.

Hope this helps.

Regards,

Neetesh

Edited by: Neetesh Raj on Sep 11, 2009 11:57 AM

[former_member187339]

Hi,

>>I am going to send a file which will be a huge file with 1GB size. What precautions should I take to process the huge file using XI? Scenario is File to IDoc. Suggest me with different options to send a huge file thourgh XI without performance issue.

Don't use file adapter with GB files.. instead try

1. breaking the file (if input file is a txt file then recordset per message can be used)

2. java proxy in sender side

3. Try http post method, where http adapter will be in the sender. (I am not sure whether this will be feasible for you or not)

Regards

Suraj