Skip to Content
avatar image
Former Member

How to reduce the amount of failed data created in Information Steward?

Hello guys,

I am developing rules to evaluate quality of data in a system. And the tables have millions of rows (6 - 15 million records). So every time i create a new rule on the table and want to evaluate the score, I end up creating huge amounts of failed data (as i have to evaluate the score for all the rules bound to that table). Also, at this point of time am not planning to create any reports from the failed data. (as am still in development phase)

I have implemented the following to reduce the amount of failed data being created;

a. Not to select the option to store failed data in 'Failed Data Repository' in the rule task.

b. Create a task to evaluate the scores after binding decent number of rules to it. (to an extent practically possible)

It would be very helpful if you can share ideas to reduce the amount of failed data being created ?


Thanks.

Add comment
10|10000 characters needed characters exceeded

  • Get RSS Feed

1 Answer

  • avatar image
    Former Member
    Oct 29, 2015 at 11:01 AM

    I think it is not a good idea to reduce amount of failed data. IS is supposed report all the data that fails. You can split it into multiple rules with filters at rule level. ex: material type, Account groups, data created time period, Order types etc..

    Add comment
    10|10000 characters needed characters exceeded