on 08-25-2016 7:05 PM
Hi,
I would like to group records based on three fields from source structure.
My Source structure:
MT_Value
----Value ( 0---Unbound)
---- ID
---- Dest
---- Source
Target also same structure but I would like to delete duplicate records with the combination of fields.
The values ID ,Dest and Source will repeated for some the records if the same combination repeats in each value node then considered that is a duplicate. How can we achieve this? I have done if it's a single field to compare duplicate records .
ID----->RemoveContext----->SplitbyValue(ValueChange)----CollapseContext----> ( Target Node ).
in my case I want to delete duplicate records based on ID, Dest and Source.
Thanks
Praveen
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Praveen,
Use below mapping.
UDF:
public void removeDuplicates(String[] id, String[] dest, String[] source, ResultList result, Container container)
throws StreamTransformationException {
List<String> list = new ArrayList<String>();
StringBuilder sb = new StringBuilder();
String key = "";
for (int i = 0; i < id.length; i++) {
if (!id[i].equals(ResultList.CC)) {
sb.setLength(0);
key = sb.append(id[i]).append(dest[i]).append(source[i]).toString();
if (list.contains(key))
result.addSuppress();
else {
list.add(key);
result.addValue("");
}
}
}
}
Testing:
Regards,
Praveen.
Hi Praveen! It's a nice shot!
Couple of thoughts in addition:
1. Adding any specific separator into key sequence will help to avoid situations like:
ID=20 Dest=BB Source=YY against ID=20B Dest=B Source=YY.
2. Transferring key building mechanism from UDF to outside mapping might help to use this function with various keys with no limits. Just to build key via source values and Concat function and pass result to UDF.
Regards, Evgeniy.
Hi Evegeniy,
Thanks for the comment.
Regards,
Praveen.
Hi Praveen,
It's really helpful....simply awesome... I tried with Udf as well as Graphical mapping with help of concat to the root node , I am able to generate unique values for the root node but in this case what ever the values under the root node are mismatching. But anyhow It resolved by your greatest code to my issue.I am really appreciate your effort.
Thank you very much...
Thanks
Praveen
User | Count |
---|---|
96 | |
11 | |
11 | |
10 | |
9 | |
7 | |
6 | |
5 | |
4 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.