BPM - Multiple input files, need wait between send of each, not working

[Former Member]

System is XI 3.0 SP19.

(Q1). Suppose there are multiple input files, the number can vary. When the BPM process triggers, the RECEIVE appears to take in all the files at the same time. There is no correlation between files; the requirement is to output / send each file (async) to the destination with a 5 minute gap between each file. (simple FTP). I have tried a BPM with a LOOP and a SEND and WAIT within the loop and a control counter, but the input files all are sent at the same time without the needed gap between files. I am new with BPM and am obviously missing something here? Suggestions please!

(Q2) With reference to Q1, suppose I receive five files, and these are being processed in a working BPM with 5 minute interval between each send, and along come three more files. Will these new arrivals wait on completion of the 1st set (this would be my expectation) or other? Same interface, etc is being used.

Thank you.

[Former Member]

Hi Mike,

If the source file of type CSV , then use the file content coversion parameters(Recordsets per Message) to process one message at a time and use a Wait step in BPM .

If the source file is an xml, Use For each block with a wait step in BPM.

Hope this might resolve the problem.

Thanks,

Kiran.

[prateek]

When the BPM process triggers, the RECEIVE appears to take in all the files at the same time.

Why don't you simply put the polling interval = 5*60 seconds? Why do you want to collect the file in BPM?

If you have to use BPM, then write a UDF and use the java function wait() to make your process flow wait for 5 minutes. Or a simple adapter module that holds the process flow.

Regards,

Prateek

[Former Member]

Prateek,

Its not about picking the input files every 5 mins. Setting the polling interval of 300 sec on sender file adapter picks the files every 5 mins and will result in the BPM producing all the files at random time intervals.

the requirement is to output / send each file (async) to the destination with a 5 minute gap between each file.

No matter how many & when they are picked, they should be output with 5 min gap between them..I'm thinking that the input files should be released to the BPM in a control manner, every 5 mins. I know we can schedule the messages with Even driven message processing, but not sure if we can set it to release only 1 msg every 5 mins.

praveen

[Former Member]

Yes, this is correct, the requirement is a 5 minute gap on the receiving side.

When there are multiple input files present at a point in time, XI retrieves all the files present. I suppose I could define multiple receive/sends in BPM but this is problematic because it creates a limit on the unknown number of files that may be present for input.

I have worked out on paper a non-BPM solution using a combination of Abap (with WAITt), Unix (to read the files in the directory 1 by 1), and can pass to XI (and in the adapter possibly use an OS command to insure the process finishes before picking up the next file) one file at a time; I would also have to control for a single instance of the program running and determine how to trigger (possibly event, scheduled, etc). This approach would not require BPM at all. A BPM solution, if it can be made to work, might be much cleaner as all the pieces are in XI.

I am attempting to work with some of the suggestions here, in particular ParForEach, and a block. There is no correlation per se between the files, but I may be able to force one (perhaps on the file name). There is really nothing in the payload that can be referenced for correlation, as it is an encrypted payload.

I do appreciate all the feedback.

[former_member206760]

i suggest coz the send is not wrkin,

Try to use the following

1. receive step with start process ticked

2. transfornation step calling a mapping which in turn contains an RFC lookup where in you have written abap code to wait for 5 min

3. async send

hope this helps

[Former Member]

Hi Mike,

Looking at the fact that all the files seem to be output at the same time..it appears to me that you are using the step Block in the 'ParForEach' mode which gets executed parallely..If that is the case you need to set your block step to 'ForEach' mode which is like serial execution of the steps inside the block.

If that is not the case, as you have not defined any correlation between the files, BPM runtime does not know for a new file picked up by the sender file adapter whether to assign the message to the existing BPM instance or create a new BPM instance. Runtime would assign to the existing BPM only in case, at the time BPM runtime received the message from IE and the BPM instace is waiting at the receive step. Otherwise it would create a new BPM instance..which will result in the case where there is not effect of the wait step.

Look at this SAP Help [Block Step|http://help.sap.com/saphelp_nw70/helpdata/en/f6/e1283f2bbad036e10000000a114084/content.htm]

[Correlating Messages|http://help.sap.com/saphelp_nw70/helpdata/en/a5/64373f7853494fe10000000a114084/content.htm]

Regd the second question, you need to ensure that BPM runtime assigns the message or wait for assigning a message to an existing BPM instance, if any. By doing so you can ensure that the wait step in the 'ForEach" mode Block gets executed and results in 5 min gap between each file produced. To do this, you need to create correlation, may be based on the File Directory & FTP source (you can get them from Adapter Specific Message Attributes). But you need to make sure that this directory is only used for this interface.

If you have input files, it leads the BPM active runtime of at least 20 minutes. If your requirement is only about producing a output file every 5 mins with simple mapping, I would rather do [Event-Driven Message Processing|http://help.sap.com/saphelp_nw70/helpdata/en/7a/00143f011f4b2ee10000000a114084/content.htm] and send the messages every 5 min to IE that simply sends to the Receiver to produce output file.

If you can send the screenshot of the BPM, it would be easy to analyze the error. I'm not sure if you can attach an image to post here..upload a flickr link.

praveen

[Former Member]

Hi,

I think, you could achieve this with 3 steps:

Receive step -> Wait Step (5 min) -> Send Step

This way, each file will start a new BPM process and the output file will be produced after 5 mins.

Be careful with the BPMs, they consume lot of resources and too many BPMs in wait status could effect your overall system performance.

And another thing you could try: setup the file adapter sender communication channel with Poll Interval to 5 mins and also change the quality of service to "Exactly Once in Order".

Hope one of these will work for you,

Sumant.

[Former Member]

Using a Receive -> Wait(5min) -> Send did not work and neither did the loop approach I tried; even when I modified the adapter to EOIO quality of service and added a queue id as suggested, no change was observed. The input files still are received together and after a 5-minute wait are sent together.

[Former Member]

Hi Mike,

Say right now you are reading from directory dir1 , have a OS script written which transfer files with a gap of five minutes and you can call this OS command in ur sender file adapter.

But i guess the wait should have worked for you... are you giving the settings correctly in the wait step. Please check.

[Former Member]

Hi, Mike:

Make sure your receive step has checked starting new process,

Make sure you did not check multi-line elements for container variable representing your source

message.

Regards.

Liang

Edited by: Liang Ji on Mar 6, 2009 10:27 PM