cancel
Showing results for 
Search instead for 
Did you mean: 

Limitation on ParForEach Mode ?

Former Member
0 Kudos

Hi Guys,

I am using the BPM with a Block step in ParForEach mode and it has a limitaion that it should be less than 999.

ParForEach Recommended

A ParForEach is recommended if:

· The messages are processed further in the parallel processing branches.

· The individual processing branches are not dependent on each other on a business level.

· There are less than 999 parallel processing branches (the multiline table element that is processed in the ParForEach contains less than 999 lines).

In my scenario if a request comes into BPM with a xml file containing 2000 transactions, i split them into each individual transaction and i process them in block in ParForEach mode.

Each transaction calls a webservice synchrously and gets the response back.

Since there is a limitaion for the no of lines for ParForEach, is there any way i can say process some transactions say 50 at a time instead of all 2000 at a time ?

Thanks,

Srini

Accepted Solutions (0)

Answers (2)

Answers (2)

moorthy
Active Contributor
0 Kudos

Hi Srinivas,

Just wonder, the no, of transactions as you mentioned is fixed here ? i.e 2k ?? because calling a WebService for each transaction in ccBPM + waiting for the acknoledgement is not a good idea... in this case. keeping performance in mind. Also executing all the mapping logic in bpm for these huge sync call is not a good idea..

Just a thought : if it is feasible:

possible to call all the transactions at a time with wrapper Service ? So the your business logic to call each transaction will reside at Target Application System.

or a create the Java Server Proxy Application, so that this will the WebService in turn and get the response at a time to XI ...

Rgds, Moorthy

Former Member
0 Kudos

Hi Krishna,

The scenario is PROXY-BPM-SOAP.

The initial request is asynchronus and dosent expect a immediate reponse.

The initial asynchr request contains a file with multiple transactions and transactions are not fixed like 2k it can be any.

Once the request is received i am splitting into each individaual transaction in BPM with a Transformation step - I cannot externalize this mapping as the BPM dosent accept the multiple messages - ( ie Multimapping with out BPM).

The next step is i am processing each transaction parallely in a Block in ParForEach mode. The block contains a synchronus send step by which each transaction calls a webservice synchronusly and receives the response.

The next step is container step where i collect all the reponse messages and merge them into a single file using another transformation step. I cannot also externalise this Transformation step.

After transforming into a single message again i make a synchronus call to another webservice and the reponse is sent back asynchronusly to the sender system using reverse proxy.

The ParForEach mode has a limitaion that the multiine table can process only 999 transactions and it will be a performance hit with transactions more than that or higher in number.

So the ideas by Raj and Liang is to have a another block step prior to the original one and this will be the first block step where it handles the transactions in ForEachMode.

Prior to this we want to modify the original first Transformation step which will split the messages in such a way that each message will have max 50 transactions. By this way the next Block step will always have not more than 50 Transactions in ParForEach mode which will be better.

Note: The webservice to the 3rd party system accepts calls for each transaction only and not for a bulk of transactions ( like a batch conataining 2k)

your idea

possible to call all the transactions at a time with wrapper Service ? So the your business logic to call each transaction will reside at Target Application System.

can you explain more, how to acheive

call all the transactions at a time with wrapper Service ?

Thanks,

Srini

Edited by: srinivas kapu on Mar 10, 2008 8:36 AM

Edited by: srinivas kapu on Mar 10, 2008 8:39 AM

Edited by: srinivas kapu on Mar 10, 2008 8:40 AM

Former Member
0 Kudos

Hi, Srini:

Just an idea for your reference. Details depend on you to figure out:

Between your mutiline elements output step (is that transformation step ?) and your ParForEach Block step,

insert two more ForEach Block Steps:

The first Block step is used to count total number of the multiline elements. Make sure this counter is on Process Level, so the value is visiable to second Block.

On your second block, you will have a switch step first,

The condition for the switch is Total Number Multiline element - Currently Processed Number.

If the result is more than 50 (saying you would like to bundle 50 msgs), you will go to branch 1, otherwise go branch 2

In branch 1, you will have loop step, loop counter end condition is 50. inside loop step, you will have one container element for append, one container element for increment loop counter. One send step after loop, then update your Currently Processed Number by Increment 50.

In branch 2, you also have a loop step, end contidion is Total Number Multiline element - Currently Processed Number. The rest of the step is same Branch one except you do not need update Currently Processed Number, since you have finished process.

Saying if you have 125 message, after above processing, you should have multiline elements: 50, 50 and 25.

Also pay attention to container element initialization, e.g.

before you get total number of the container element, set it to 0 first.

Regards !

Liang

Edited by: Liang Ji on Mar 8, 2008 7:12 PM

justin_santhanam
Active Contributor
0 Kudos

Liang,

I also thought the same, but little bit different logic. I don't know how does it works, but logically it shud work. See my below logic and share ur thoughts in it.

Srini,

Keep all ur objects as it is. Create one more mapping the source and target must be same which u used before Par-For-Each Block.

The new mapping program must split the transactions 50 ,50 instead of your each transaction.

Now the BPM logic.

New Mapping Program -


>For-Each Block -


> Old Mapping Program----


>Par-For-Each Block

Now ,let say u have 200 transactions.

- The new mapping program will split 50,50,50,50 - Hence 4 Messages will be there.

- For-Each Block take First Message(50) and come inside the block. Now your old mapping program will split the 50 transaction into single transaction. Now your Par-For-Each block will get executed.

- Once the Par-For-Each block completes it's exexcution, again it will come to For-Each Block. Now the Second set of 50 transactions will go inside, the process continues..........

I hope it should work, work on it and share ur experiences too!

raj.

Former Member
0 Kudos

Hi, Raj:

Great idea ! 100% agree with you !

Actually, I had same idea to make some tricks on mapping program.

Rather than making new mapping as you mentioned, I was thinking about modify old maping program directly to produce multiline elements with number of 50, 50....

I believe too, your logic will work for sure, and just need less modification on existing process.

Srini, you should try this one first

Regards

Liang

Former Member
0 Kudos

Hi Liang and Raj,

Thanks for the suggestions and quick reply. I will try this logic and let you know. I know your ideas will work 100%.

Thanks,

Srini