File Adapter for picking multiple files in sequence based on condition

[Former Member]

Hi PI Experts,

In our scenario, SAP BW is writing 3 files in one of NFS location.

.dat file ( Which contains the records or data )

.man file ( Which contains total number of records in .dat file, run data and time of .dat file etc )

.done file ( This is the empty file, which means other two files .dat and .man are completed and its ready to pickup )

SAP PI need to pick the above files only if the .done file is written in NFS location.

Currently we are using the Integrated Configurations, and the SAP PI picks the .dat file before .done file is written in NFS.

We can increase the scheduling time but not sure how much time SAP BW takes to write the files.

Some .dat files sizes are more then 3GB and it takes several hours to write the files.

So is there any way we can specify that the other files needs to be picked up in sequence based on the existance of one file.

like in our case PI has to pick .dat file from the folder only if .done file is available.

and it should pick in the below in sequence as

.dat

.man

.done

Thanks,

Kiran.

[Former Member · Accepted solution]

Hi Kiran,

You mentioned that .done file is an empty file. Will you be picking up this file as by default file adapter doesnt pick up empty file or it wont generate the message for it. If you generate the msg for it, it will give strucuture error.

File adapter doesnt provide the standard functionality of dependancy on file where a particular file is chekced first & if available, then only start the processing.

One option could be to write a custom JAVA module which will check if .done file exist. if yes then only process the files else terminate the processing.

Other option could be use BPM where in BPM will wait till .done file comes & then it will generate the target message in sequence as required.

Regards

Sushil

[Former Member · Accepted solution]

Hi Sushil,

Thanks for suggesting to go with Custom JAVA Module.

We are back to go with this solution, could you please provide us the logic to solve the issue.

pick the files in sequence based on the condition.

Sequence

.dat

.man

.done

Condition

Pick .dat file only if the adapter finds .done file in the NFS location.

Sushil or any one can provide the required logic for custom JAVA module,

hope fully that should solve the issue and I can close long pending post.

Thanks every one who suggested multiple options.

Thanks

Kiran.

[Shabarish_Nair · Accepted solution]

I wonder how this would be done in a module.

use a script.

The same was mentioned in your earlier post.

PS: Could you please close this post and maintain one single post.

[Former Member · Accepted solution]

Thanks Vijay, I am closing this thread to maintain one. But as requested we would be looking for JAVA module or some other option, not the script.

Thanks,

Kiran.

[justin_santhanam]

Kiran,

Take a look at this -

I don't know if it helps- but it can lead you to different ideas.

raj.

[Former Member]

Hi Raj / Raj,

The two answers are helpful to some extent, but problem is till not solved as

my case here is slightly different.

- we wanted to avoid script and PI is able to transfer 3GB file with out splitting file.

Here is my case -

The files to be picked up by file adapter will be written to folder one by one in below sequence.

First .dat, then .man and finally .done.

I just wanted to make sure that the folder has .done file before i start picking files.

Because I want to make sure writing of file to .dat is fully completed as the .dat file size will be huge.

but if i just keep the Quality of service as EOIO-- Exactly Once in Order.

and processing Mode be BY NAME or BY DATE, it doesn't really help me,

because it takes time to finish writing .dat file to the folder and at the same time

if the file adapter looks for file sequence then there will not be any other files other than .dat (still be in process of writing to it).

so it simply picks the .dat file even other system doesnu2019t complete writing the whole content to it..

Though we want the .dat to be picked first, we cannot do it before .done is placed in the folder because the writing of data to the .dat file will not be completed.

Thatu2019s the only reason we wanted to check for .done file before picking the .dat file.

Because the other system writes .dat file first and once it completes writing then it writes .man file and once that is also completed then it finally writes .done file to the folder.

Let me put this in other way.

I want my file adapter to pick the .dat file once the file is completely written.

i don't want to put the huge poll interval in file adapter parameters instead i want my adapter to check if file is completed before it start picking of any file from the folder.

Or it has to look for .done file before it start picking file.

I hope i am clear, please let me know if my question is still not clear.

Thanks,

Kiran.

[justin_santhanam]

Kiran,

The other possibility I can think of is using a BPM? Are you ok with it?

raj.

[justin_santhanam]

Kiran,

The other possibility I can think of is using a BPM? Are you ok with it?

raj.

[Former Member]

Hi Raj,

No .. we cannot use that, its an integrated scenario because we just wanted to FTP

we are not doing any mapping etc.

I am just looking for something to be done at file adapter level.

Thanks,

Kiran.

[Former Member]

Hi,

we cannot use that, its an integrated scenario because we just wanted to FTP

we are not doing any mapping etc.

If u r nt using any mapping means.... then i think the sequence of processing the files doesnt matter...(of course it depends upon the target adapter and business requirements)...

Shall we know what u r doing with all these files.. writing into tables or just to other FTP location??????

so let the File adapter process the files in any sequence.... and for only the data file u need to handle that it shldn nt be picked in middle...

U can change the extension of file while writing say .tmp and once finished it is .dat file.....

Lemme know ur view on this..

Babu

[Former Member]

Hi,

So is there any way we can specify that the other files needs to be picked up in sequence based on the existance of one file.

Yes, there is a way

Use scripts to handle this. That will be the best solution.

Some .dat files sizes are more then 3GB and it takes several hours to write the files. 

Just out of curiosity, how you are handling the file as big as this one? I hope you are splitting up the file.

Regards,

Neetesh