Remove Duplicate Records

[gagandeep_batra]

Hi,

I am working on File to File scenario...I need to remove the duplicate records.

I am also refer the following thread but it's not working can someone help me...

http://scn.sap.com/thread/999261

<Records>

<Row>

<ID>10001</ID>

<Name>Praveen</Name>

<Age>22</Name>

<Gender>M</Gender>

</Row>

<Row>

<ID>11901</ID>

<Name>Raj</Name>

<Age>24</Name>

<Gender>M</Gender>

</Row>

<Row>

<ID>10001</ID>

<Name>Praveen</Name>

<Age>22</Name>

<Gender>M</Gender>

</Row>

</Records>

Target

<Records>

<Row>

<ID>10001</ID>

<Name>Praveen</Name>

<Age>22</Name>

<Gender>M</Gender>

</Row>

<Row>

<ID>11901</ID>

<Name>Raj</Name>

<Age>24</Name>

<Gender>M</Gender>

</Row>

</Records>

Regards

Gagandeep

[baskaran_k2 · Accepted solution]

Hi,

This is very simple and can handle it in mapping,

ID->Sort->SplitbyValue(Value change)->collapseContexts ->SplitbyValue(EveryValue)->ID

Regards

Baskaran K

[gagandeep_batra · Accepted solution]

HI Baskaran,

I am getting ID is correctly by writing the UDF but rest of the field context is miss match. like name fields

Regards

GAgandeep

[former_member184681 · Accepted solution]

Hi,

You would need a UDF to control the creation of the Row node (execution type: all values of queue, single input parameter input😞

for (int i = 0; i < input.length; i++) {

    int found = 0;

    for (int j = 0; j < i; j++) {

       if input[i].equals(input[j]) {

          found = 1;

          break;

      }

    }

    if (found > 0) result.addSuppress();

    else result.addValue(input[i]);

}

Then map as follows:

ID(context: Records) -> UDF -> Row

Finally, for the Row subnodes (fields) including ID, use simple 1:1 mapping.

Regards,

Greg

[Former Member · Accepted solution]

no need to write any UDF , have to try below logic:

1. ID -> remove context -> Sort -> Split by value(value change) -> id

2. ID -> remove context ->     Sort By key -> Split by value(value change) -> name

name -> remove context ->

same second logic  for remining all fields(age , gender) ..

id -> remove context -> sort -> Split by value(value change) -> exits -> collpcecontext -> createif -> Row

[gagandeep_batra · Accepted solution]

Hi Glowacki

i use following

ID(context: Records) -> Remove Context -> UDF -> Row

then it working thanks for ur help.

Regards

Gagandeep

[zameerf · Accepted solution]

Greg,

Finally, for the Row subnodes (fields) including ID, use simple 1:1 mapping.

I guess you missed splitByValue(EachValue) in between source and target field, for field level mapping.

Edited:

sorry, i overlooked that .. it should work with 1:1 map

[iaki_vila]

Hi Gagandeep,

You could try with this XSL-Mapping:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" version="1.0" encoding="UTF-8" indent="yes"/>
    <xsl:template match="@*|node()">
        <xsl:if test="not(node()) or not(preceding-sibling::node()[.=string(current())])">
            <xsl:copy>
                <xsl:apply-templates select="@*|node()"/>
            </xsl:copy>
        </xsl:if>
    </xsl:template>
</xsl:stylesheet>

I try with this example:

<?xml version="1.0" encoding="UTF-8"?>
<Records xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\ivm72g\Escritorio\Entrada.xsd">
    <Row>
        <ID>10001</ID>
        <Name>Praveen</Name>
        <Age>22</Age>
        <Gender>M</Gender>
    </Row>
    <Row>
        <ID>11901</ID>
        <Name>Raj</Name>
        <Age>24</Age>
        <Gender>M</Gender>
    </Row>
    <Row>
        <ID>10001</ID>
        <Name>Praveen</Name>
        <Age>22</Age>
        <Gender>M</Gender>
    </Row>
    <Row>
        <ID>10001</ID>
        <Name>Praveen</Name>
        <Age>22</Age>
        <Gender>M</Gender>
    </Row>
    <Row>
        <ID>11901</ID>
        <Name>Raj</Name>
        <Age>24</Age>
        <Gender>M</Gender>
    </Row>
</Records>

I obtain this output:

<?xml version="1.0" encoding="UTF-8"?>
<Records xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="C:\Documents and Settings\ivm72g\Escritorio\Entrada.xsd">
    <Row>
        <ID>10001</ID>
        <Name>Praveen</Name>
        <Age>22</Age>
        <Gender>M</Gender>
    </Row>
    <Row>
        <ID>11901</ID>
        <Name>Raj</Name>
        <Age>24</Age>
        <Gender>M</Gender>
    </Row>
</Records>

I hope this works correctly always.

Regards.

[anupam_ghosh2]

Hi Iñaki Vila,

                    awesome XSL code. Could you please explain the test statement and in general what your code is doing. That will be of great learning.

Regards

Anupam