I have a scenario wherein I will have to load the Master Data from a flat file. The load has to happen quite frequently. I have a flat file wherein one of the fields in my flat file has to be loaded to the Dimension ID of one Dimension. The flat file contains multiple records with the same ID. I used transformation file to load the data from a flat file. So when I tried to load the data, it throws me an error that the duplicate records are detected.
Does anyone have any idea how we can skip the duplicate records using transformation file options and mapping section?