Remove Duplicate values

[former_member625050]

Hi All,

  The following is the input structure

<ContextChange>

10

abc

asd

qwe

<ContextChange>

20

qwe

ert

hjk

<ContextChange>

10

lkj

cvb

rfg

<ContextChange>

now basing on the first field , output should be segregated like following

<ContextChange>

10

abc

asd

qwe

lkj

cvb

rfg

<ContextChange>

20

qwe

ert

hjk

<ContextChange>

the all values under the same qualifier should come under one context.Kindly help.

Regards,

[Former Member · Accepted solution]

Hi Irfan,

Please find the below mapping logic for your requirement.

Regards,
Bhavin

[former_member182412]

Hi Shaik,

Use below UDF.

public void joinQueue(String[] queue, ResultList result, Container container) throws StreamTransformationException {
  SortedMap<String, List<String>> map = new TreeMap<String, List<String>>();
  boolean isPreviousCC = false;
  String key = "";
  for (int i = 0; i < queue.length; i++) {
  if (isPreviousCC || i == 0) {
  key = queue[i];
  isPreviousCC = false;
  continue;
  }
  List<String> list = map.get(key);
  if (list == null) {
  list = new ArrayList<String>();
  list.add(queue[i]);
  map.put(key, list);
  } else {
  if (!queue[i].equals("__cC_"))
  list.add(queue[i]);
  }
  if (queue[i].equals("__cC_"))
  isPreviousCC = true;
  }
  boolean first = true;
  for (Entry<String, List<String>> entry : map.entrySet()) {
  List<String> list = entry.getValue();
  if (!first)
  result.addContextChange();
  else
  first = false;
  result.addValue(entry.getKey());
  for (String value : list) {
  result.addValue(value);
  }
  }
  }

Regards,

Praveen.