big File to IDOC - performance issue

[Former Member]

Hi All,

I am trying to create scenario where I have a file with aproximately 10 000 rows. From each row I am creating one IDOC and want to send it to R/3. Interface looks fine - it is working, but it is killing XI box for some time and u cant access it.

Full scenario look like this

File -> BPM (for 1:n) -> IDOC

I tried to find some solutions for doing the workload smaller by splitting file to less lines (500rows per file) but then file adapter picks up all file and processed them in parallel. So this is new scenario:

BigFile -> XI -> File -> BPM(1:n) -> IDOC

I tried to put second file sender communication channel as EOIO but looks like this does not work - or messages from queue are processed to fast. When one message starts BPM another file message start to be processed.

Do You have any ideas on how to make it more responsive and less performance impact?

thanks in advance.

Dawid

[Former Member · Accepted solution]

Try to increase the heap size

Mudit

[Former Member · Accepted solution]

Hi ;

Since mappings are processed by the J2EE Engine, the maximum available Java heap may be a limit-ing factor for the maximum document size the XI mapping service is able to process. Tests have shown that processing of XSLT mappings consumes up to 20 times the source document size (using identity mapping). The maximum available Java heap for 32bit JVMs is platform-dependent. Using 64bit JVM platforms is an option here.

Current maximum heap sizes – 32bit

OS

Maximum heap (GB)

Linux

2

Windows

1.2 – 1.4

The Java heap is limited by the heap limit of the process (may be limited by address space because operating system code or libraries may also be loaded within the same address space). Also, Java internal memory areas such as the permanent space for loading Java classes must fit into the same address space.

Java VM tuning is one of the most crucial tuning steps, especially for more complex scenarios. For information about setting baseline JVM parameters, see SAP Note 723909. You must also take plat-form-specific parameters into account (for example, JIT compiler settings). The impact of Garbage Collection (GC) behavior especially may become a critical issue. Overall GC times for the J2EE appli-cation should be well below 5%. For more information about GC behavior and settings, see also SAP Note 552522.

Specific to XI is the fact that you sometimes need to process large documents for mapping or when using signatures. This can lead to excessive memory usage on the Java side. Therefore, you must observe Garbage Collection and the available Java heap in order to evaluate performance and pre-vent OutOfMemory exceptions. Since XI mapping is processed by stateless session beans that are called using a JCo interface, this may lead to a reduction of parallel JCo server threads within the JCo RFC Provider service of a J2EE server node (you can compensate for this by adding J2EE server nodes).

Mudit

[Former Member · Accepted solution]

Hi Mudit,

I am looking for solution more on my side - not basis side. How to solve problem during developing phase that I can setup interface to be more performance wise(:. Any other ideas?

br

Dawid

[Former Member · Accepted solution]

As the file size is huge i cant think of any thing else other that the tunning on the basis side.

Mudit

[Former Member]

Thanks all,

I will try idoc bundilng and get rid of BPM.

[Former Member]

Hi Dawid,

Why do you use a BPM for this scenario? You can avoid that and there by increase performance atleast 25%. Just change the occurance of IDOC in exported IDOC XSD and import it back and use it in mapping. This is called IDOC bundling, explained by Mr. XI here /people/michal.krawczyk2/blog/2005/12/04/xi-idoc-bundling--the-trick-with-the-occurance-change

Good Luck.

Regards,

Jai Shankar

[prateek]

Try removing BPM from ur scenario.

1:n could be achieved without BPM

/people/narendra.jain/blog/2005/12/30/various-multi-mappings-and-optimizing-their-implementation-in-integration-processes-bpm-in-xi

Regards,

Prateek

[Former Member]

Hi Prateek,

Unfortunately 1:n without BPM is not possible for idoc adapter and http adapter. So I am limited here. Or I am missing something.

br

Dawid