Skip to Content
0

Files getting picked up early from SAP PI directory and from the source directory

Dec 26, 2017 at 05:48 PM

116

avatar image
The inbound (file to IDoc scenario) is such that:

1. Legacy application would place file (with some specific file name schema)in in its specified file folder.
2. IM would pick the file from the above folder with the help of some event and place file in folder /abc
3. PI would raise the above event with the help of specific job(run once a day) and hence it picks the file and place file from /abc and place in X3 folder /xyz
3. PI would pick the file from /xyz of bussiness service to PI X3 Server.
4. PI(X3) would receive the IDoc using Message.IDoc.Basictype and process it and generated iDoc

Message.IDoc.Basictype

5. IDoc Message.IDoc.Basictype would be posted in ECC systems.

I am facing problem from the 4th step onwards, incomplete file is getting received by PI and thus failing at PI processing level.

Can anyone help me what level do I need to investigate to find tha actual root cause or why is this happening?

Please come up with your valuable suggestions and discussions.

10 |10000 characters needed characters left characters exceeded
* Please Login or Register to Answer, Follow or Comment.

3 Answers

Best Answer
Ryan Crosby Jan 01 at 11:26 PM
0

Hi Aishvar,

Based on what you have described it sounds like your file sender polling is running too frequently (probably the default of 60 seconds which IMO is way too often) and you have not handled the potential for file writing & reading/deletion collision occurring by using the file modification check. The other possible method to solve the problem is to dial back the file polling frequency to decrease the chances of such a collision from occurring, which can also be further enhanced with the channel availability option.

Regards,

Ryan Crosby

Show 2 Share
10 |10000 characters needed characters left characters exceeded

Thanks a lot Ryan for your valuable reply.

My current sender polling frequency is 60 seconds, and I will use the functionality of file modification check in this case.

As per the trend of last two months, I have figured out the size of the file which PI is picking around 1MB.

Taking these factors into consideration, how much polling do you suggest in this case?

Any other way you suggest for handling it other than file modification check?

Could you please elaborate more on channel availability option.

Thanks in advance.

Your thoughts on this would be really helpful and appreciated.

Let me know if any further input you require from my end.

Regards

Aishvar

0

Hi Aishvar,

I would guess that the total file writing time for 1MB does not take very long but when you are polling so frequently you have a higher probability that your PI system will poll right as the file is in the process of getting created. We had an instance of this where invoices were getting picked up while the file was incomplete and causing the contents to never get sent (I don't care for the 60 second default because if it needs to get sent that often it probably shouldn't be file based). If you dial back the time to say 5, 10, 15 minutes or more then you start decreasing that probability with each step. The question is related to how critical this asynchronous integration is for the business and how long you can go without sending the data once it is ready. The channel availability option allows you to designate schedules for the channel so that when the schedule is not active the channel will be "stopped" which means it is not polling. You could use this option to set a specific schedule during each day, week or month for the channel to be running (polling).

Regards,

Ryan Crosby

0
Aishvar Keshari Dec 27, 2017 at 11:49 AM
0

Please let me know if you need more clarity in understanding the query

Share
10 |10000 characters needed characters left characters exceeded
avatar image
Former Member Dec 27, 2017 at 12:07 PM
0

Hi,

Can you tells us, is file at /xyz location is XML or csv and do u check the format of file?

Regards,

Anoop Rai

Show 7 Share
10 |10000 characters needed characters left characters exceeded
The file at /xyz location is XML only, there is no content conversion configured.

I did check the format is XML everywhere both at /xyz and /abc

Thanks for your valuable reply.

Please let me know if anything else needed from my side.

0
Former Member

Hi,

As per your inputs, XML file is picked by PI and generate a IDoc that will push to ECC System. If its correct check below:-

i. XML tags in file are same as PI XML Tags.

ii. IDoc related settings have been done at PI end, specially updated metadata has been imported in PI System.

iii. Use test configuration in ID, and put File data in Payload and check whether result is proper or not.

Regards,

Anoop Rai.

0

The problem here that I am facing is :

PI is picking incomplete file from /xyz and hence posting at X3 Server

This can be confirmed while monitoring in NWDI (failing in X3) ,due to incomplete XML structure (if we tally it with actual incoming file placed in /abc, source directory ).

I believe there is an overlap in the processing of Job and PI processing.

PI is configured to delete the file once it gets processed, but in this case I can see PI is picking-processing and deleting files before the job scheduled tasks are completed.

That means it is processing incomplete file and then deleting it from its end.

And this deletion is occurring before the last job step is executed.

We have kept a poll interval of 60s in this case, but I don't think changing the poll interval could be of use in this case.

Suggestions please

0
Former Member
Aishvar Keshari

Hi,

Please use modification check in the sender file channel which is used to pick up files from /xyz folder.

I guess this channel is picking files when the file was not written completely.

Thanks,

Apu

1

Thanks for your valuable answer Apu..

Would like to ask what can be the root cause for this..

Thoughts please..

0
Former Member
Aishvar Keshari

There can be only one reason of partial reading of file using sender communication.

When your second interface is picking up the file, first interface is not yet completed the writing of the file. So, if you use modification check in the sender CC of the 2nd interface then the sender CC of the second interface will wait to pick up the file until first interface will complete the file write.

Thanks,

Apu

1
Thanks for the interaction Apu.

I am aware of the functionality Modification check and I am using the same on my interface channel level.

What I want to know is why is this happening ?

At OS level, there might be scripts written, on which 1st ID is for job processing and then PI ID comes into picture

Once PI ID is processed which triggers the functionality of file deletion after processing the file.

Need to investigate on what basis is this script triggering this deletion functionality..does it check for complete file? or it has any time constraint after which it executes this functionality..

If it checks for complete file, is there any concept of locks which is used ? means it cant delete the complete file once it is completely written.

Your thoughts would be valuable and appreciated..

0