how to handle very high volume msgs in XI

[Former Member]

I have a scenario where 6 million msgs/ hour(xml) have to be picked form source and to be sent to target ,

How do we handle this high volume of data is there any good practice

Pls suggest .

[deviprasad_pothireddy]

Hi raj,

Also check this blog

<a href="/people/alessandro.guarneri/blog/2007/02/21/sap-xi-acting-as-a-huge-file-mover XI acting as a (huge) file mover</a>

[Former Member]

Hi Anubhav,

if you just have to send source file to target system without doing any mapping, then it dont use XI for it.......because you have a very high volume of msgs and for this such load, XI will start putting your msgs in wait status and may be the JAVA engine may go down because of insufficient memory.......

So better have an FTP utility in target system.....connect to source system by this...and move the files from source to target by this FTP utility......

But if you have to do a mapping from source msg to target msg, then use XI....but try doing the mapping by graphical mapping and dont go for ABAP mapping to reduce memory usage for your scenario.......

Thanks,

Rajeev Gupta

[Former Member]

Weare not using FTP here rather websphere MQ is being used...any suggestion how MQ can help on this.

[Former Member]

Hi Anubhav,

so you are using MQ ......see in MQ you have queues.......and every queue is configured to take input from some directory.....so just configure a queue in MQ for a source directory and then put all yur source files in this directory, then MQ will pickup and move your file......

Thanks,

Rajeev Gupta

[Former Member]

Dear Rajeev,

Will MQ handle such a high volume of data 6 mil/day if so how pls guide on configuring this .

[Former Member]

MQ can handle this volume because it stores all data in queue and sends one by one if the message is small is 2kb are so then you use XI because the XI gets one message at a time and processing that should not be a problem . but make sure you havecluster of XI servers to process other request.

Regards

Sreeram.G.Reddy

[Former Member]

Dear Sriram ,

What do you mean cluster of XI servers ,do you mean more than one can you comment on the suggested landscape.

[Former Member]

Please use group of server to handle such large voulme of data i.e. you need to have 4 to 5 instance of XI to do load balancing of such large vpolume of data .

Regards

Sreeram.G.Reddy

[Former Member]

Which is a better middleware solution SAP XI or Websphere MQ for handling large volume of data.Is it possible to use both SAP XI and Websphere MQ,MQ for queing msgs and handling the same msgs in XI,any best practice ???

[Former Member]

for large volumeuse MQ. we can use both . best practise is use MQ only

[former_member431549]

The two products are not similar in functionality. XI is a middle-ware and MQ is a physical transport method. XI can use MQ as the transport. The key is whether you just need guaranteed delivery and no transformation, then use MQ.

[Former Member]

Dear Tim ,

Thats what i want XI to make use of MQ as transport ,but will it be able to handle high volume of data (6 mil/day) if so then how ? any best practices you could suggest.

[Former Member]

Anubhav ,

XI can handle 6 mil per day . as isaid you need to havecluster of server for permonace point of view to diretubethe load between diffrent XI servers because in paralle we will have many other XXI interface running.

Regards

Sreeram.g.reddy

[Former Member]

Hi,

>> have a scenario where 6 million msgs/ hour(xml) have to be picked form source and to be sent to target ,

This is not a recommended approach by SAP.

Instead of sending many small msgs, try to club them into msgs of bigger size and send them. SO that the number of msg processed gets reduced to say 6000/hour....

Can you give more details about yuor sender and reciver?

Regards,

Jai Shankar

[Former Member]

Sender system are global,they do postings on router system which XI has to pick and send to R/3,no. of msg are more because we receive from all locations globally,

we cannot group this msg into smaller groups as it is coming in batch.