cancel
Showing results for 
Search instead for 
Did you mean: 

How to reduce the amount of failed data created in Information Steward?

Former Member
0 Kudos

Hello guys,

I am developing rules to evaluate quality of data in a system. And the tables have millions of rows (6 - 15 million records). So every time i create a new rule on the table and want to evaluate the score, I end up creating huge amounts of failed data (as i have to evaluate the score for all the rules bound to that table). Also, at this point of time am not planning to create any reports from the failed data. (as am still in development phase)

I have implemented the following to reduce the amount of failed data being created;

a. Not to select the option to store failed data in 'Failed Data Repository' in the rule task.

b. Create a task to evaluate the scores after binding decent number of rules to it. (to an extent practically possible)

It would be very helpful if you can share ideas to reduce the amount of failed data being created ?


Thanks.

Accepted Solutions (0)

Answers (1)

Answers (1)

former_member655569
Participant
0 Kudos

I think it is not a good idea to reduce amount of failed data. IS is supposed report all the data that fails. You can split it into multiple rules with filters at rule level. ex: material type, Account groups, data created time period, Order types etc..