Eliminating of duplicate records

[Former Member]

Hi,

My source structure look like

Node1

--- Field1

--- Field2

--- Field3

--- Node2

-- Field4

-- Field5

-- Field6

How to eliminate the duplicate records from the above source structure of six fields of combination.I have tried with in graphical mapping but unable to do it.

Can any one provide a sample udf.

Thanks in advance

Regards

Tiger Woods

[Former Member]

Hi

Thanks for your replies,

The given UDF is looking for the duplicate values to one field only,I want to eliminating the duplicate values for all the combination of all fields values.For ex we have the data like

1,2,3,4,5,6

10,20,30,40,50,60

1,2,3,4,5,7

10,20,30,40,50,60

From the above test data we have three unique data records 4th record should be ignore.

Regards

Tiger Woods

[Former Member]

Hi

I am not a java expert,could you please provide a UDF for this requirement..,

Regards

Tiger Woods

[Former Member]

HI,

i can give you for one field try to use it for your actual requirement..

see if input is field1 with multiple values then see the below UDF

field1removecontextUDF -- Splitbyvalue--target node

in UDF -- select type as queue

Vector vfield1 = new Vector();

for(int i = 0; i , field1.length;i++){

if(!vfield1.contains(field1(i))) vfield.add(field1(i));

}

Collections.sort(vfield);

for(int j =0;vfield.size;j++){

result.addValue((String)vfield.get(i));

}

Regards

Chilla

<i>reward points if it is helpful..</i>

[Former Member]

HI,

Suppose , if you have the above structure under root node with multiple occurrences ,

root --- 0..unbound

Node1

--- Field1

--- Field2

--- Field3

--- Node2

-- Field4

-- Field5

-- Field6

So write a UDF and map to root to target root,

In UDF

take all into a Vector,

and while adding into the vector check the record structure is there or not if it is not existed the add to result

Regards

Chilla

<i>reward points if it is helpful..</i>