Delete a flat file using script in BODS

[krishnamohan_corp9]

Hi,

I have 9 target files in my network drive/folder. out of which 2 are having data. other 7 files dont have data except headers.

can any one help me how to delete those empty files which are having only headers from the network folder.

How to acheive this using BODS?

Any help appreciated.

Regards

Krishna Mohan

[bittu_sharma]

Hi ,

Help!!

can we delete files older than x days using BODS inbuilt function-

file_delete( '\\\\sharedpath\\test_file\\*.txt');

or any other option to delete files older than x days using any BODS inbuilt function.

Regards,

Bittu

[moshenaveh]

bittu_sharma

Hello, While we're happy that you've come to SAP Community to get an answer to your question, you posted your question as an answer in an old thread.
I've converted your answer to a comment, but even so -- posting in older threads is not the best way to get guidance.
If you're looking for help, you should ask a new question: https://answers.sap.com/questions/ask.html.
Here are some tips to help you craft an effective question for our community: https://community.sap.com/resources/questions-and-answers, https://developers.sap.com/tutorials/community-qa.html, https://groups.community.sap.com/t5/welcome-corner-discussions/advice-from-sap-champions-questions-a....
I encourage you to follow this guidance, as I'd really like to see you get a solution to your problem.
I hope you find this advice useful!

[krishnamohan_corp9 · Accepted solution]

Steps for deleting the files with no data in outbound folder:

Steps 1: Declare variables $G_TOTAL_ROWS (data type as int), $G_VALUE (data type as int)

Step2:  create a script after the dataflow where you are creating the multiple files.

Step3: Write the below script under script_file_deletion

$G_TOTAL_ROWS = total_rows('DATASTORENAME.DBO.FIRSTTABLENAME);

print('Total records for FIRST_TABLE is :');

print($G_TOTAL_ROWS);

IF ($G_TOTAL_ROWS < 1)

$G_VALUE=exec('sh', '-c "rm /Data/FILE_NAME1.txt"', 8);

#Print('Data does not exist for FILE_NAME1 so file is deleted');

else

print('Data Exists for FILE_NAME1);

the above code works for LINUX not for windows.

Step4: Repeat code for other files.

Step5: Check whether file is deleted or not in the folder

Note: above script works for Linux platform.

Regards

Krishna Mohan

[Former Member]

Hi,

Can you try below syntax,it might be helpful to you .Give the flat file path in YOur shared path

exec('cmd','del \\\ your shared path\archive\mpi\*.xml',8);

Thanks

[krishnamohan_corp9]

the syntax which you have provided will only delete a file from ur drive, i need to check a file which has no data (file may have headers), i need to delete only those files.

[Former Member]

You can use following script which I have mentioned earlier for identifying the size of the file & then deleting:

If SAP Data Services is hosted on Linux/AIX you can write shell script

if [ `wc -l filename` -gt 0]

rm filename

If SAP Data Services is hosted on Windows, you can write bat file

FINDSTR /R /N "^.*$" filename.txt | FIND /C ":"

You have to store file names in a file and then use while loop to read the file name one by one.

Thanks

[former_member234353]

Hi Krishna,

Hope the following reference will help you fulfill your requirement.

How to avoid empty file generation in BODS | SCN

Hope this helps.

And if you need any specifics regarding implementing for your scenario..please do let me know

Thanks,

AJ

[vnovozhilov]

TS mentioned that the requirement is to delete "those empty files which are having only headers". I read it as "non empty" files, therefore the approach suggested above would not work in this case.

The approach would remain the same:

Check tall files in a loop (any method of preference, DS script or shell script will do).

Use "IF (condition) THEN (action)" within that loop.

The action is already known -- to remove the files which satisfy the condition.

All you need to do is to formalize what is the criteria to find "those empty files which are having only headers".

Thank you,

Viacheslav.

[former_member234353]

Hello,

I agree with your point that I provided with a solution that would help identify empty files in comparison to non-empty files.

But the point I wanted to make that is using the file size of get_file_attribute() function and that files lower than a particular threshold can be deleted.

Another solution that I had in mind was the use of exec() command to find the number of lines present in each of the files.If the file contains only one line, it means that its a header file and with no data. Therefore the above condition if met would be deleted. Else the file has data along with headers.

$L_num_lines = exec('/bin/sh',-c'wc -l <$filename>',2);

if($L_num_lines <= 1)

exec('/bin/sh',-c'rm <$filename>',2);

else

<continue with the next file>;

Thanks,

AJ

[vnovozhilov]

Sure, my intention was to reply to all the above, not specifically to your post. I guess "wc -l" would be a feasible option here.

Thank you,

Viacheslav.

[krishnamohan_corp9]

Hi anirban,

I have a source, case transform, a query and multiple target files, where should i define those conditions and all using script. could u please elaborate.

Regards

Krishna

[former_member234353]

Hello Krishna,

If I can deduce your requirement correctly from different replies that you have posted, you are looking to implement similar to the following req.

1. Extract data from source.

2. Identify data and according to a case logic segregate them into different files.

3. Irrespective of which files have the data, all the files would have their headers generated.

Now you need to delete the files that don't produce any data for that specific run and this goes on for all the delta runs as well.

If the above case seems relevant then insert a script at the end of the dataflow and you will be left with the files that have data along with header and the remaining files that don't have the data will be removed.

Hope this helps.

Thanks,

AJ

[krishnamohan_corp9]

Hi anirban,

Yes.  That's what I have done.  Please check my comment for correct answer.  Thanks for helping. 

Regards

Krishna

[former_member198401]

Krishna

It will get complicated if you write the logic to delete the empty excel files. Data Services can process multiple excel files and you should not worry about the empty ones.

You can include DI_FILENAME for each excel file. The excel containing the data will be loaded and the DI_FILENAME will be populated.

Hope this simplifies your work

Regards

Arun Sasi

[Former Member]

Hello Krishna,

If SAP Data Services is hosted on Linux/AIX you can write shell script

if [ `wc -l filename` -gt 0]

rm filename

If SAP Data Services is hosted on Windows, you can write bat file

FINDSTR /R /N "^.*$" filename.txt | FIND /C ":"

You have to store file names in a file and then use while loop to read the file name one by one.

[aasavaribhave]

write a windows bat file to check if excel file with header is empty. Call the bat file in exec() in BODS.