Why Data Services Validation doesn't give the right result in XML?

[isathore]

Hi,

I am struggling with a dataflow that contains the following:

- an Excel as source

- a validation transform that check a specific value in one of the excel columns. If the value is equal, the record goes to the pass path and if not, it goes to fail path

- a simple query on the pass path with a row_gen that selects all passed records and send it to an XML file (using XML target)

- a CSV target file on the fail path that gets the failed record from the validation

The result is unexpected: both files contain the failed records.

Now, if you modify slightly the dataflow and add another pass path to the validation, which has a CSV file as target, suddenly, the result is what you would expect:

- The XML file on the 1st pass path (with the query) now contains the successful records

- The CSV file on the 2nd pass path contains the successful records as well

- The CSV file on the fail path contains the failed records

If you create a dataflow that doesn't have an XLM file as a target, this issue doesn't seem to happen. I have made several examples with datastore and with CSV files. Somehow it seems there is something happening wrongly when the only target for a validation path (pass or fail) is an XML file.

I can send the ATL for both examples above

Could someone help me understand if I do something wrong OR if there is indeed a bug that should be addressed?

Thanks a lot!

Isabelle

[isathore]

Hi Manoj,

Thanks for quick reply!

To answer your questions:

- in the validation transform what is the value of Action on failure option - send to fail, send to pass or both ?

send to fail only

- is delete and recreate option checked for the target xml file ?

yes

- how many rows do you see in monitor log for pass and fail ?

In the 1st dataflow, the monitor show the following where the pass number is 26!

Path name State Row Count Elapsed time (Sec) Absolute time (Sec)

DFW_TransformationUsingValidation_5/SpinFormat_1 STOP 28 0.047 0.699

/DFW_TransformationUsingValidation_5/Validation STOP 28 0.000 0.699

/DFW_TransformationUsingValidation_5/Split STOP 26 0.000 0.699

-DFW_TransformationUsingValidation_5/FailValidation_TransformationUsingValidation_5.csv STOP 26 0.000 0.699

/DFW_TransformationUsingValidation_5/Row_Generation STOP 1 0.000 0.699

/DFW_TransformationUsingValidation_5/CacheSplit STOP 26 0.000 0.699

/DFW_TransformationUsingValidation_5/CacheSplitMemoryReader STOP 26 0.000 0.699

/DFW_TransformationUsingValidation_5/Validation_Pass_1 STOP 26 0.000 0.699

/DFW_TransformationUsingValidation_5/Query-Nest1: 0 STOP 1 0.047 0.699

/DFW_TransformationUsingValidation_5/Query-Nest1: 1 STOP 26 0.000 0.699

/DFW_TransformationUsingValidation_5/Query STOP 1 0.000 0.699

-DFW_TransformationUsingValidation_5/GUID::'5e0a492d-1452-4e0b-be37-758c2249b7a4' LOAD MESSAGE PassXM STOP 1 0.000 0.699

In the 2nd dataflow, just by adding another pass path filling a CSV file, the number of pass records is 2:

DFW_TransformationUsingValidation_5/SpinFormat_1 STOP 28 0.047 0.689

/DFW_TransformationUsingValidation_5/Validation STOP 28 0.000 0.689

/DFW_TransformationUsingValidation_5/Split STOP 2 0.000 0.689

-DFW_TransformationUsingValidation_5/PassValidation_TransformationUsingValidation_5.csv STOP 2 0.000 0.705

-DFW_TransformationUsingValidation_5/FailValidation_TransformationUsingValidation_5.csv STOP 26 0.000 0.705

/DFW_TransformationUsingValidation_5/Row_Generation STOP 1 0.000 0.705

/DFW_TransformationUsingValidation_5/CacheSplit STOP 2 0.000 0.705

/DFW_TransformationUsingValidation_5/CacheSplitMemoryReader STOP 2 0.000 0.705

/DFW_TransformationUsingValidation_5/Validation_Pass_1 STOP 2 0.000 0.705

/DFW_TransformationUsingValidation_5/Query-Nest1: 0 STOP 1 0.047 0.705

/DFW_TransformationUsingValidation_5/Query-Nest1: 1 STOP 2 0.000 0.705

/DFW_TransformationUsingValidation_5/Query STOP 1 0.000 0.705

-DFW_TransformationUsingValidation_5/GUID::'5e0a492d-1452-4e0b-be37-758c2249b7a4' LOAD MESSAGE PassXM STOP 1 0.000 0.705

Edited by: Isabelle Thore on Oct 8, 2010 6:53 AM

Edited by: Isabelle Thore on Oct 8, 2010 6:54 AM

Edited by: Isabelle Thore on Oct 8, 2010 6:55 AM

[Former Member]

I will have to look a the DF

can you post this in other DI/DS forum which allows you to attach smaller files

http://www.forumtopics.com/busobj/viewforum.php?f=83

post this message under Data Integrator or DI: Designer and Job Design sub-forum under it

[isathore]

Hi,

Kindly find the ATL and related files in the post [http://www.forumtopics.com/busobj/viewtopic.php?p=762639#762639] on the BusinessObject Board forum.

Thanks & Best Regards

Isabelle

Edited by: Isabelle Thore on Oct 11, 2010 5:03 AM

[isathore]

Hi,

This is a bug and was addressed to SAP development.

Follow the BoB thread. http://www.forumtopics.com/busobj/viewtopic.php?t=164394&start=0

Best Regards

Isabelle

[Former Member]

what is the version of Dataservices ?

in the validation transform what is the value of Action on failure option - send to fail, send to pass or both ?

is delete and recreate option checked for the target xml file ?

how many rows do you see in monitor log for pass and fail ?