Dear All,
We have a batch job which is scheduled to be run at midnight and load source xml files to our data warehouse. The source files are sent to our DI server from external source system. There could be 100~200 source file per day and each of them has same layout but different name (with extraction timestamp in the file name, e.g. order_20120101073003). The existing batch job use wildcard in the source file name to process all files in one go.
Now we are asked to convert it to a real-time job so that the source files can be processed as soon as they arrived. We have no experience in real-time environment. I read the "Building Data Integrator Real-time Jobs and Calling Web Services", but still have some questions:
1. It seems like to use XML file as source for real-time job, the file name has to be consistent, and the job will be kick-off when change is detected in the file. Is it correct?
2. It saids the real-time job has to return an XML message as output. However, our job is to populate a table in relational database. What's the purpose of having a return XML message and how can it be configured?
3. Is there anything else you can advise regarding how can we convert this job to real-time?
Thanks!
Knight