on 06-11-2012 4:43 PM
Hello SAP Gurus,
I have a file-to-file scenario .
The source file is provided in Unix server where once it gets written , the file is not getting renamed or deleted.
One more twist is that this file with content has a check if there is a similar file with ".OK" extension(which is empty).
Lets take an example :
Directory : /usr/interface/test has two files :
ABC.ok is without content
ABC is with content.
"ABC. ok" gets populated by ABAP program, in the end when the entire file is written , so that we do not have the possibility of picking the file in the middle.
Now i have to write a Adapter module to check following things :
1) if " ABC.OK " exists then pick up the actual file with content
2) Delete the "ABC.OK" file
3) Move the actual file to archive folder for future reference.
Could you please help in this .
Regards,
Ravi
Hi Ravi,
I have described working with trigger files in PI in one of my Wikis below, have a look and let me know if you still have some questions:
http://wiki.sdn.sap.com/wiki/display/XI/Sender+File+Adapter+Frequently+Asked+Questions
Regards,
Greg
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hello Grzegorz,
Thanks a lot for your helpful blog.
The Fourth question in your blog gives me an approach for my solution. But i have two question :
1) Will this work for filenames like " ABC_* " . Bcoz i have 5 files during the day coming from Application server and they all have to be treated in the same way.
2) At the end, i would like to delete both the files from the source directory .
" ABC_* " and " ABC_*.OK " as well.
I can set the deletion of " ABC_* " file by setting the communication channel setting .
But how about deletion of " ABC_*.OK " file.?
Your quick response is very much appreciated.
Regards,
Ravi
Dear Ravi,
For this purpose, use the following configuration (assuming your original file's extension is .xml, if not, change accordingly):
For file deletion, simply set Processing Mode = Delete in the Processing tab page. The additional file should be deleted together with the main file automatically by PI.
Regards,
Greg
Dear Ravi,
For FTP, it gets more complicated, since you cannot use the option mentioned. Instead, you can:
1. Develop a custom adapter module that will check if the trigger file exists.
2. Use the Lookup API to check the trigger file during mapping in a UDF, and throw Smart Exceptions in case it isn't there.
Regards,
Greg
Hello Greg ,
I have actually tried and written a code.
The scenario is that it checks for "ABC.OK" file in the directory . If it finds then it executes the code.
But the problem is that the actual file ABC ( with content ) is not getting deleted from this code.
Also pls note that i am using fclient because, while testing we got the FTP timeout error. so we are connecting with the FTP client via code.
But unfortunately, this fclient is not having a class for deletion of file and so my entire effort has gone for a toss.
/**
*
*/
package com.sap.module;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.rmi.RemoteException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import javax.ejb.EJBException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import javax.ejb.TimedObject;
import javax.ejb.Timer;
import sun.net.ftp.FtpClient;
import com.sap.aii.af.lib.mp.module.ModuleContext;
import com.sap.aii.af.lib.mp.module.ModuleData;
import com.sap.aii.af.lib.mp.module.ModuleException;
import com.sap.engine.interfaces.messaging.api.Message;
import com.sap.engine.interfaces.messaging.api.MessageKey;
import com.sap.engine.interfaces.messaging.api.Payload;
import com.sap.engine.interfaces.messaging.api.PublicAPIAccessFactory;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditAccess;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditLogStatus;
public class GetMaterialInformationBean implements SessionBean, TimedObject {
public ModuleData process(ModuleContext moduleContext, ModuleData inputModuleData)
throws ModuleException {
try{
AuditAccess audit = null;
MessageKey key = null;
Object obj = null;
Message msg = null;
String host = null;
String uname = null;
String pwd = null;
String archiveDir = null;
int read;
Calendar currentDate = Calendar.getInstance();
SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd");
String dateNow = formatter.format(currentDate.getTime());
obj = inputModuleData.getPrincipalData();
msg = (Message) obj;
//Values to be maintained as channel parameters
host = moduleContext.getContextData("host");
uname = moduleContext.getContextData("uname");
pwd = moduleContext.getContextData("password");
archiveDir = moduleContext.getContextData("archiveDir");
if (host == null) {
throw new ModuleException(
"Error in module: Module Parameter \"host\" not specified");
}
if (uname == null) {
throw new ModuleException(
"Error in module: Module Parameter \"uname\" not specified");
}
if (pwd == null) {
throw new ModuleException(
"Error in module: Module Parameter \"password\" not specified");
}
if (archiveDir == null) {
throw new ModuleException(
"Error in module: Module Parameter \"archiveDir\" not specified");
}
Payload p = msg.getMainPayload();
try {
key = new MessageKey(msg.getMessageId(), msg.getMessageDirection());
audit = PublicAPIAccessFactory.getPublicAPIAccess()
.getAuditAccess();
} catch (Exception e) {
ModuleException me = new ModuleException(e);
throw me;
}
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"enteredModule");
String okfileName = msg.getMessageProperty("http://sap.com/xi/XI/System/File","FileName");
String dirName = msg.getMessageProperty("http://sap.com/xi/XI/System/File","Directory");
int index = okfileName.indexOf(".");
String fileName = okfileName.substring(0,index);
String arcFileName = fileName + dateNow;
msg.setMessageProperty("http://sap.com/xi/XI/System/File","FileName",fileName);
//audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"mainfileName" + fileName);
String fileLocation = dirName+ "/" + fileName;
String archiveLocation = archiveDir+"/" +arcFileName;
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"mainfileLocation=" + fileLocation);
FtpClient fClient = new FtpClient();
fClient.openServer(host);
fClient.login(uname,pwd);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"connection established");
fClient.ascii();
/*For picking file from Server
File f1 = new File(fileLocation);
boolean success = f1.exists();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"success=" + success );
if( !success ){
throw new Exception("File is not available");
}
byte[] buffer = new byte[(int)f1.length()];
FileInputStream f = new FileInputStream(fileLocation);
f.read(buffer);
p.setContent(buffer);
msg.setMainPayload(p);
inputModuleData.setPrincipalData(msg); */
//Read data from the file FtpClient way
InputStream in = fClient.get(fileLocation);
//long length = in.available();
BufferedReader BR = new BufferedReader(new InputStreamReader(in));
StringBuilder SB = new StringBuilder();
String line1 = null;
while ((line1 = BR.readLine()) != null) {
SB.append(line1 + "\n");
}
BR.close();
String data = SB.toString();
byte[] buffer = data.getBytes();
//in.read(buffer);
p.setContent(buffer);
msg.setMainPayload(p);
inputModuleData.setPrincipalData(msg);
//Write file to archive directory and delete file
OutputStream out = fClient.put(archiveLocation);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"archivefileLocation=" + archiveLocation);
out.write(buffer);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"" +
"written to archive file");
//boolean resp = fClient.deleteFile(fileLocation);
//audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"File deleted, response" + resp);
//last steps
out.close();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"out is closed");
fClient.closeServer();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"server connection is closed");
return inputModuleData;
fClient.
}catch(Exception ex)
{
ModuleException me = new ModuleException(ex);
throw me;
}
}
How about using this code to delete the file?
http://www.kodejava.org/examples/358.html
It looks reasonable
Regards,
Greg
Well, right, I didn't notice that. However, this example with your particular class looks promising: http://www.nsftools.com/tips/SunFtpWrapper.java. Using this issueCommand("DELE " + fileName) method, you should be able to delete the file.
Regards,
Greg
Hi,
Even I had a similar requirement, I went to additional file option, where I check for ".ok" file existence in the folder and then pick the original file.
Note: This option only works in NFS and does not work when you use FTP parameters.
You may need a Adapter module when u use FTP option.
Regards,
Sainath Chutke
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Ravi,
This has to be done by Basis team only. In case if you are using FTP parameters then you need to write a adapter module.
Hope this solves your issue
The code required is :
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import javax.ejb.CreateException;
import com.sap.aii.af.service.auditlog.;
/**
@ejbHome <{com.sap.aii.af.mp.module.ModuleHome}>
* @ejbLocal <{com.sap.aii.af.mp.module.ModuleLocal}>
* @ejbLocalHome <{com.sap.aii.af.mp.module.ModuleLocalHome}>
* @ejbRemote <{com.sap.aii.af.mp.module.ModuleRemote}>
* @stateless
* @transactionType Container
/
import com.sap.aii.af.mp.module.;
import com.sap.aii.af.ra.ms.api.Message;
import java.util.Hashtable;
import java.io.;
import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
import java.net.;
import java.util.List;
import java.util.ArrayList;
public class CustomFileBean implements SessionBean, Module {
public void ejbRemove() {}
public void ejbActivate() {}
public void ejbPassivate() {}
public void setSessionContext(SessionContext context) {
myContext = context;
}
private SessionContext myContext;
public void ejbCreate() throws CreateException {}
public ModuleData process(ModuleContext moduleContext, ModuleData
inputModuleData) throws ModuleException{
Object obj = null; // Handler to get Principle data
Message msg = null; // Handler to get Message object
String fileName = null; // File Name which is being processed
URL FileLocation;
obj = inputModuleData.getPrincipalData();
msg = (Message) obj;
Hashtable mp = (Hashtable)
inputModuleData.getSupplementalData("module.parameters");// Get the Supplemental data which is available as hash table
if (mp != null) fileName = (String)mp.get("FileName"); // Get the file name which is being processed
AuditMessageKey amk = new AuditMessageKey(msg.getMessageId(),AuditDirection.INBOUND);// to put the message in the audit log
String Location = moduleContext.getContextData("Location"); // moduleContext is used to retrieve data supplied by user
String FtpUser = moduleContext.getContextData("FtpUser"); // these data will be extracted from the module configuration of the channel
String PassLocation = moduleContext.getContextData("PassLocation");// which is passed as parameter value
//this code will read names of files which has been already processed and convert them into array so that it can be easily compared
//with the next name of file
List wordList = new ArrayList();
BufferedReader br = null;
try {
br = new BufferedReader( new FileReader(Location) );
String word;
// loop and read a line from the file as long as we dontget null
while( ( word = br.readLine() ) != null )
// add the read word to the wordList
wordList.add( word );
}
catch(IOException ex){
ex.printStackTrace();
}
finally {
try {
// attempt to close the file
br.close();
} catch( IOException ex ) {
ex.printStackTrace();
}
}
try{
// initialize a new string array equal to the size of the wordList
String[] words = new String[ wordList.size() ];
// call the wordList's toArray method to and transfer itemsfrom
// wordList to our string array words
wordList.toArray( words );
Writer output = null;
File file = new File(Location);
output = new BufferedWriter(new FileWriter(file));
int flag = 0;
for( int i = 0; i < words.length; i++ )
{
String text = words[i]+"\r\n";
output.write(text);
if(words[i].equals(fileName))
flag=1;
}
// if file has not been processed earlier
if (flag==0)
{
//write the name file to the list
output.write(fileName);
output.close();
}
// if the file has been processed earlier
else
{
output.close();
try{
BufferedReader Read = new BufferedReader(new FileReader(PassLocation)); //to read the password of the FTP server
Audit.addAuditLogEntry(amk, AuditLogStatus.ERROR,"File has been already processed");
FileLocation = new URL("ftp://"+FtpUser+":"+Read.readLine()+"@"+msg.getMessageProperty("http://sap.com/xi/XI/System/File","SourceFTPHost")+ msg.getMessageProperty("http://sap.com/xi/XI/System/File","Directory")+"/"+fileName+ "");
//msg.getMessageProperty("http://sap.com/xi/XI/System/File","SourceFTPHost") this is used to retrieve the name of host. For this adapter specific message attributes must be enabled in the channel.
DeleteFile(FileLocation); //to delete the file
}
catch (Exception e)
{
throw new ModuleException(fileName+"should be deleted");
}
throw new ModuleException(fileName+"is already processed");
}
}
catch( Exception e ) {
ModuleException me = new ModuleException(e);
throw me;
}
return inputModuleData;
}
public void DeleteFile(URL FileLocation) throws Exception{
URLConnection uc = FileLocation.openConnection() ; // to establish connection with FTP server using URL
uc.setDoOutput(true);
OutputStream out = uc.getOutputStream();
out.flush();//to make the file empty
out.close();//close the file so that connection can be terminated
}
}
Hello Sainath,
I have actually tried and written a code.
The scenario is that it checks for "ABC.OK" file in the directory . If it finds then it executes the code.
But the problem is that the actual file ABC ( with content ) is not getting deleted from this code.
Also pls note that i am using fclient because, while testing we got the FTP timeout error. so we are connecting with the FTP client via code.
But unfortunately, this fclient is not having a class for deletion of file and so my entire effort has gone for a toss.
/**
*
*/
package com.sap.module;
import java.io.BufferedReader;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.rmi.RemoteException;
import java.text.SimpleDateFormat;
import java.util.Calendar;
import javax.ejb.EJBException;
import javax.ejb.SessionBean;
import javax.ejb.SessionContext;
import javax.ejb.TimedObject;
import javax.ejb.Timer;
import sun.net.ftp.FtpClient;
import com.sap.aii.af.lib.mp.module.ModuleContext;
import com.sap.aii.af.lib.mp.module.ModuleData;
import com.sap.aii.af.lib.mp.module.ModuleException;
import com.sap.engine.interfaces.messaging.api.Message;
import com.sap.engine.interfaces.messaging.api.MessageKey;
import com.sap.engine.interfaces.messaging.api.Payload;
import com.sap.engine.interfaces.messaging.api.PublicAPIAccessFactory;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditAccess;
import com.sap.engine.interfaces.messaging.api.auditlog.AuditLogStatus;
public class GetMaterialInformationBean implements SessionBean, TimedObject {
public ModuleData process(ModuleContext moduleContext, ModuleData inputModuleData)
throws ModuleException {
try{
AuditAccess audit = null;
MessageKey key = null;
Object obj = null;
Message msg = null;
String host = null;
String uname = null;
String pwd = null;
String archiveDir = null;
int read;
Calendar currentDate = Calendar.getInstance();
SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd");
String dateNow = formatter.format(currentDate.getTime());
obj = inputModuleData.getPrincipalData();
msg = (Message) obj;
//Values to be maintained as channel parameters
host = moduleContext.getContextData("host");
uname = moduleContext.getContextData("uname");
pwd = moduleContext.getContextData("password");
archiveDir = moduleContext.getContextData("archiveDir");
if (host == null) {
throw new ModuleException(
"Error in module: Module Parameter \"host\" not specified");
}
if (uname == null) {
throw new ModuleException(
"Error in module: Module Parameter \"uname\" not specified");
}
if (pwd == null) {
throw new ModuleException(
"Error in module: Module Parameter \"password\" not specified");
}
if (archiveDir == null) {
throw new ModuleException(
"Error in module: Module Parameter \"archiveDir\" not specified");
}
Payload p = msg.getMainPayload();
try {
key = new MessageKey(msg.getMessageId(), msg.getMessageDirection());
audit = PublicAPIAccessFactory.getPublicAPIAccess()
.getAuditAccess();
} catch (Exception e) {
ModuleException me = new ModuleException(e);
throw me;
}
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"enteredModule");
String okfileName = msg.getMessageProperty("http://sap.com/xi/XI/System/File","FileName");
String dirName = msg.getMessageProperty("http://sap.com/xi/XI/System/File","Directory");
int index = okfileName.indexOf(".");
String fileName = okfileName.substring(0,index);
String arcFileName = fileName + dateNow;
msg.setMessageProperty("http://sap.com/xi/XI/System/File","FileName",fileName);
//audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"mainfileName" + fileName);
String fileLocation = dirName+ "/" + fileName;
String archiveLocation = archiveDir+"/" +arcFileName;
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"mainfileLocation=" + fileLocation);
FtpClient fClient = new FtpClient();
fClient.openServer(host);
fClient.login(uname,pwd);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"connection established");
fClient.ascii();
/*For picking file from Server
File f1 = new File(fileLocation);
boolean success = f1.exists();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"success=" + success );
if( !success ){
throw new Exception("File is not available");
}
byte[] buffer = new byte[(int)f1.length()];
FileInputStream f = new FileInputStream(fileLocation);
f.read(buffer);
p.setContent(buffer);
msg.setMainPayload(p);
inputModuleData.setPrincipalData(msg); */
//Read data from the file FtpClient way
InputStream in = fClient.get(fileLocation);
//long length = in.available();
BufferedReader BR = new BufferedReader(new InputStreamReader(in));
StringBuilder SB = new StringBuilder();
String line1 = null;
while ((line1 = BR.readLine()) != null) {
SB.append(line1 + "\n");
}
BR.close();
String data = SB.toString();
byte[] buffer = data.getBytes();
//in.read(buffer);
p.setContent(buffer);
msg.setMainPayload(p);
inputModuleData.setPrincipalData(msg);
//Write file to archive directory and delete file
OutputStream out = fClient.put(archiveLocation);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"archivefileLocation=" + archiveLocation);
out.write(buffer);
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"" +
"written to archive file");
//boolean resp = fClient.deleteFile(fileLocation);
//audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"File deleted, response" + resp);
//last steps
out.close();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"out is closed");
fClient.closeServer();
audit.addAuditLogEntry(key, AuditLogStatus.SUCCESS,"server connection is closed");
return inputModuleData;
fClient.
}catch(Exception ex)
{
ModuleException me = new ModuleException(ex);
throw me;
}
}
Hello Ravi,
Are you open to making changes to the ABAP program that writes the file and the file.OK?
Instead of writing a file.OK at the end of writing the original file, it can simple rename the original file right?
like while writing the file, name it as temp.txt and atthe end, rename it to ABC.txt.
This way you can do away with the adapter modules and java code.
Best Regards,
Ravikanth Talagana
Just few cents... If your requirement is so generic and usable for different interfaces you might still consider creating adapter module. But IMO, since the requirement is just to make a call to OS and do the following things like list the files in a directory and check whether desired file exists or not, delete the files and moving the desired file to the specified directory. All these simple operations can be well handled with few lines of codes in script. So I would go for writing script for this case. If you need to write those commands you can search online for writing those unix commands.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I dont recommend Adapter module , handle using Schell scripts .
Try Anupam Approach.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Ravi,
Its possible to write a adapter module to put all functionality you have mentioned, though I never wrote one like this one earlier. While looking for solutions without adapter module I thought of this one
1. use a sender communication channel to pick up file only with name ABC after regular intervals of time.
2. Do an FCC if reqd and let the data flow into a mapping.
3. In the mapping have an UDF as shown below. This UDF has following functions
a. Check if there is a file (created on current date) in the same folder from where ABC has been
picked up.
b. Check if the name of the file in step a is ABC.ok or not.
c. If answer to step b is "yes" process the file in further mappings. If answer step b is "no"
then raise an exception to stop the mapping further.
I feel this scenario is feasible to achieve. The way to stop a mapping is shown here in this article http://scn.sap.com/people/alessandro.guarneri/blog/2006/01/26/throwing-smart-exceptions-in-xi-graphi...
In case you do not want to stop the mapping with an exception then depending on the value returned by the UDF in step 3 (say 0-success,1- failure) , the second mapping will either execute or add error trace in sxmb_moni as shown in this thread http://scn.sap.com/thread/1904565 .
so interface is
pick file ABC---->map1 with UDF---->check return value in map2 ----> add trace in sxmb_moni or
proceed with creating the target file.
Regards
Anupam
Hi Ravi,
I feel u can use shell scripts like this
while [ 1 -gt 0 ] # infinite loop
do
if [ -f "ABC.OK" ] ; then # check in current folder if ABC.Ok exists or not
mv ABC /usr/interface/test/inbound #move the file ABC to /usr/interface/test/inbound folder
rm -f ABC.OK #delete ABC.OK
fi
sleep 1h # sleep for one hour the Unix sleep command
done
The script runs in infinite loop in folder /usr/interface/test. checks for abc.ok and puts ABC to inbound folder from where PI picks it up. Then abc.ok is deleted . script runs after every 1 hour.
Regards
Anupam
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
85 | |
10 | |
10 | |
9 | |
6 | |
6 | |
6 | |
5 | |
4 | |
3 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.