cancel
Showing results for 
Search instead for 
Did you mean: 

RSA3 - Extractor shows duplicate records

Former Member
0 Kudos

Hi All,

We have created one generic extractor based function module, to extract data from file path AL11.

Issue: If we trigger infopackage in BW end, it fetch records according to the source (Source file). Whereas we have checked the extractor in RSA3, it shows more enteris.

Ex. Source file have 4 records.

If we triggered infopackage, it extracts only 4 records.

If we checked RSA3, it shows 40 records (based on Data extr. call).ie.,Data extr. call = 10, No of records =10*4 = 40.

Kindly give your input.

Thanks,

Angavai Elangovan

Accepted Solutions (0)

Answers (3)

Answers (3)

Former Member
0 Kudos

Hi All,

Thanks for your quick response.Below is the piece of code.

FUNCTION xxxxxxxxx.

*"----------------------------------------------------------------------

*"*"Local Interface:

*"  IMPORTING

*"     VALUE(I_REQUNR) TYPE  SRSC_S_IF_SIMPLE-REQUNR

*"     VALUE(I_DSOURCE) TYPE  SRSC_S_IF_SIMPLE-DSOURCE OPTIONAL

*"     VALUE(I_MAXSIZE) TYPE  SRSC_S_IF_SIMPLE-MAXSIZE OPTIONAL

*"     VALUE(I_INITFLAG) TYPE  SRSC_S_IF_SIMPLE-INITFLAG OPTIONAL

*"     VALUE(I_READ_ONLY) TYPE  SRSC_S_IF_SIMPLE-READONLY OPTIONAL

*"     VALUE(I_REMOTE_CALL) TYPE  SBIWA_FLAG DEFAULT SBIWA_C_FLAG_OFF

*"  TABLES

*"      I_T_SELECT TYPE  SRSC_S_IF_SIMPLE-T_SELECT OPTIONAL

*"      I_T_FIELDS TYPE  SRSC_S_IF_SIMPLE-T_FIELDS OPTIONAL

*"      E_T_DATA STRUCTURE  XXXXXX OPTIONAL

*"  EXCEPTIONS

*"      NO_MORE_DATA

*"      ERROR_PASSED_TO_MESS_HANDLER

*"----------------------------------------------------------------------

** Data Declaration

TYPES:

* type declaration to hold file data

  BEGIN OF ty_file1,

    sarchitecture TYPE ZARCH_FROM_SAP,           

    splant   TYPE zplant1,                      

    spartner TYPE ZBU_PARTNER1,                 

    spartnername type ZPARTNER_NAME,            

    uarchitecture type ZARCH_TO_USF,            

    uplant   TYPE zplant2,                      

    upartner TYPE ZBU_PARTNER2,                 

    upartnername type ZPARTNER_NAME,            

    datum    TYPE Zdate,                        

    Flag     TYPE Zflag,                        

  END OF ty_file1,

  t_file1 TYPE STANDARD TABLE OF ty_file1.

*Declaration to hold the data from XXXXXX table

  TYPES:

  BEGIN OF ty_data,

    partner   TYPE BU_PARTNER,

    territory TYPE CRMT_TERRITORY_ID,

  END OF ty_data.

*Declaration for BUT000 table

TYPES:

  BEGIN OF ty_but000,

    partner      TYPE BU_PARTNER ,

    partner_guid TYPE BU_PARTNER_GUID,

  END OF ty_but000.

*Declaration for CRMD_TERR_ACCREL Table

TYPES :

  BEGIN OF ty_extract,

    partner_guid TYPE BU_PARTNER_GUID,

  END OF ty_extract.

  TYPES : BEGIN OF L_BUT100,

           PARTNER TYPE BU_PARTNER,

           RLTYP TYPE BU_PARTNERROLE,

          END OF L_BUT100.

*Declaration of internal table for holding the file data and logical reading of system

  DATA : pc_it_file1 TYPE t_file1,

         pv_file1    TYPE localfile.

  DATA:

    lv_line  TYPE char240,

    ls_file1 TYPE ty_file1,

   sysname  TYPE SBIWM_LOGSYS,

    client(3)   TYPE C,

    cnt  TYPE i.

DATA:

  it_file1        TYPE t_file1,

  it_data         TYPE STANDARD TABLE OF ty_extract,

  wa_data1        TYPE ty_data,

  It_but000       TYPE STANDARD TABLE OF ty_but000,

  Wa_but000       TYPE ty_but000,

  wa_data_extract TYPE zcust_user,

  LT_BUT100 TYPE TABLE OF L_but100,

   LT_BUT000 TYPE TABLE OF ty_BUT000.

* Auxiliary Selection criteria structure

  DATA: l_s_select TYPE srsc_s_select.

* Maximum number of lines for DB table

  STATICS: s_s_if TYPE srsc_s_if_simple,

* counter

          s_counter_datapakid LIKE sy-tabix,

* cursor

          s_cursor TYPE cursor.

* Select ranges

  RANGES: l_r_partner FOR but000-partner.

* Initialization mode (first call by SAPI) or data transfer mode

* (following calls) ?

  IF i_initflag = sbiwa_c_flag_on.

************************************************************************

* Initialization: check input parameters

*                 buffer input parameters

*                 prepare data selection

************************************************************************

* Check DataSource validity

    CASE i_dsource.

      WHEN 'XXXXXXXXX'.

      WHEN OTHERS.

        IF 1 = 2. MESSAGE e009(r3). ENDIF.

* this is a typical log call. Please write every error message like this

        log_write 'E'                  "message type

                  'R3'                 "message class

                  '009'                "message number

                  i_dsource   "message variable 1

                  ' '.                 "message variable 2

        RAISE error_passed_to_mess_handler.

    ENDCASE.

    APPEND LINES OF i_t_select TO s_s_if-t_select.

* Fill parameter buffer for data extraction calls

    s_s_if-requnr    = i_requnr.

    s_s_if-dsource = i_dsource.

    s_s_if-maxsize   = i_maxsize.

* Fill field list table for an optimized select statement

* (in case that there is no 1:1 relation between InfoSource fields

* and database table fields this may be far from beeing trivial)

    APPEND LINES OF i_t_fields TO s_s_if-t_fields.

  ELSE.                 "Initialization mode or data extraction ?

************************************************************************

* Data transfer: First Call      OPEN CURSOR + FETCH

*                Following Calls FETCH only

************************************************************************

* First data package -> OPEN CURSOR

    IF s_counter_datapakid = 0.

* Fill range tables BW will only pass down simple selection criteria

* of the type SIGN = 'I' and OPTION = 'EQ' or OPTION = 'BT'.

*      LOOP AT s_s_if-t_select INTO l_s_select WHERE fieldnm = 'PARTNER'.

*        MOVE-CORRESPONDING l_s_select TO l_r_partner.

*        APPEND l_r_partner.

*      ENDLOOP.

* Determine number of database records to be read per FETCH statement

* from input parameter I_MAXSIZE. If there is a one to one relation

* between DataSource table lines and database entries, this is trivial.

* In other cases, it may be impossible and some estimated value has to

* be determined.

      OPEN CURSOR WITH HOLD s_cursor FOR

*      SELECT TERR_GUID TERR_ID FROM CRMM_TERRITORY

*                               WHERE TERR_ID LIKE 'T%'.

  SELECT PARTNER RLTYP FROM BUT100

                               WHERE

                                   ( RLTYP = 'Z001' OR RLTYP = 'Z002' ).

ENDIF.                                "First data package ?

* Fetch records into interface table.

*   named E_T_'Name of extract structure'.

    FETCH NEXT CURSOR s_cursor

              APPENDING CORRESPONDING FIELDS

               OF TABLE LT_BUT100

               PACKAGE SIZE s_s_if-maxsize.

  IF sy-subrc <> 0.

      CLOSE CURSOR s_cursor.

    RAISE no_more_data.

   Else.

*    CHECK LT_BUT100[] IS not INITIAL.

*     if LT_BUT100[] is not initial.

*        SELECT PARTNER PARTNER_GUID

*          FROM BUT000

*          INTO TABLE LT_BUT000

*          FOR ALL ENTRIES IN LT_BUT100

*          WHERE PARTNER = LT_BUT100-PARTNER.

*        SORT LT_BUT000 BY PARTNER.

*      endif.

CALL FUNCTION 'RSA_LOGICAL_SYSTEM_BIW_GET'                " Function module which dynamically choose system and client

*   EXPORTING

*     I_WITHOUT_POPUP                = ' '

   IMPORTING

     E_LOGSYS_BIW                   = sysname

*   EXCEPTIONS

*     NO_LOGSYS_TO_THIS_CLIENT       = 1

*     LOGSYS_CHANGED                 = 2

*     OTHERS                         = 3

            .

  IF SY-SUBRC <> 0.

* Implement suitable error handling here

  ELSE.

    client = sysname+7(3).

    sysname = sysname+0(3).

  ENDIF.

  CONCATENATE '/usr/app/sap/' sysname '/'

              client '/SM/conversion/USF/XXXXXXXXXXX.txt'

  INTO pv_file1.

*  pV_file1 = '/usr/app/sap/SC1/200/SM/conversion/XXXXXXXXXX.txt'.

  CLEAR cnt.

  OPEN DATASET pv_file1 FOR INPUT IN TEXT MODE ENCODING DEFAULT.

  IF sy-subrc IS NOT INITIAL.

*   No access to read the Cust Plant XREF application server file

*    MESSAGE i000 WITH text-i11.

  ELSE.

    DO.

      READ DATASET pv_file1 INTO lv_line.

      IF sy-subrc = 0.

        cnt = cnt + 1.

        IF cnt NE 1.

        SPLIT lv_line AT cl_abap_char_utilities=>horizontal_tab

         INTO ls_file1-sarchitecture

              ls_file1-splant

              ls_file1-spartner

              ls_file1-spartnername

              ls_file1-uarchitecture

              ls_file1-uplant

              ls_file1-upartner

              ls_file1-upartnername

              ls_file1-datum

              ls_file1-Flag.

        CALL FUNCTION 'CONVERSION_EXIT_ALPHA_INPUT'        "Function which converts the customer number to 10 digit

          EXPORTING

            INPUT         = ls_file1-spartner

         IMPORTING

           OUTPUT        = ls_file1-spartner

                  .

        APPEND ls_file1 TO pc_it_file1.

        ENDIF.

      ELSE.

        EXIT.

      ENDIF.

      ENDDO.

    CLOSE DATASET pv_file1.

ENDIF.

  SORT pc_it_file1 BY spartner.

DELETE ADJACENT DUPLICATES FROM pc_it_file1 COMPARING spartner.

"Comparison of data from the file with the Account realtionship table

IF pc_it_file1[] IS NOT INITIAL.

   SELECT partner

         partner_guid

    FROM BUT000

    INTO TABLE it_but000

     FOR ALL ENTRIES IN pc_it_file1

   WHERE partner = pc_it_file1-spartner.

endif.

IF it_but000[] IS NOT INITIAL.

   SELECT partner_GUID

     FROM CRMD_TERR_ACCREL

     INTO TABLE it_data

      FOR ALL ENTRIES IN it_but000

    WHERE partner_guid = it_but000-partner_guid.

endif.

   SORT it_data BY partner_guid.

  Delete adjacent duplicates from it_data comparing partner_guid.

LOOP AT it_but000 INTO wa_but000.

   READ TABLE it_data

      WITH KEY partner_guid = wa_but000-partner_guid

      BINARY SEARCH TRANSPORTING NO FIELDS.

   IF NOT SY-SUBRC EQ 0.

     wa_data_extract-sold_to = wa_but000-partner.

     wa_data_extract-territory = space.

     APPEND wa_data_extract TO E_T_DATA.

    ENDIF.

  ENDLOOP.

SORT E_T_DATA BY SOLD_TO TERRITORY.

      DELETE ADJACENT DUPLICATES FROM E_T_DATA COMPARING SOLD_TO TERRITORY.

ENDIF.

   s_counter_datapakid = s_counter_datapakid + 1.

   ENDIF.             "Initialization mode or data extraction ?

ENDFUNCTION.

sander_vanwilligen
Active Contributor
0 Kudos

Hi Angavai,

Can you please give some high level information on the program logic? What are you trying to accomplish?

I do not understand the coding completely. In the first line of your discussion I read that you are using this Function Module Extractor to extract from a Flat File. I can see indeed in the lower part of the coding statements like OPEN DATASET.

However, in the first part of the coding I can see the statement

      OPEN CURSOR WITH HOLD s_cursor FOR

*      SELECT TERR_GUID TERR_ID FROM CRMM_TERRITORY

*                               WHERE TERR_ID LIKE 'T%'.

  SELECT PARTNER RLTYP FROM BUT100

                               WHERE

                                   ( RLTYP = 'Z001' OR RLTYP = 'Z002' ).

I need some functional explanation to fully understand what you are doing here.

However, it might be that with this constellation the Flat File is potentially read more than once, i.e. once per data package. I don't know if that is on purpose and if this explains the different behavior between RSA3 (with more than one data package) and BW extraction (with possibly only one package).

Thanks,

Sander

Former Member
0 Kudos

Hi Sander,

Thanks for your response. Actually i have commented that part of open cursor code.Basically i am trying to read the data from the file in AL11 server,which i am placing it in Internal table.After which the i am comparing the data which i got from the internal table with a standard table (CRMD_TERR_ACCREL). So data from the internal table should not be present in the standard table. These data will be placed in the Extract structure (E_T_DATA). This is the functionality of the code.


So initially, when the code was working fine,i was not able to extract the data to BW. That is infopackage is running for a longtime and when i check the job log in Source system side,it shows a dump "TSV_TNEW_OCCURS_NO_ROLL_MEMORY". But the idocs generated by the job were transferred to BW successfully. So this error which i am facing at present.So i tried to modify the function module to overcome this.Please let me know,how to resolve this issue.


Regards

Angavai

sander_vanwilligen
Active Contributor
0 Kudos

Hi Angavai,


I would not recommend your approach if the flat file data is your starting point. OPEN CURSOR etc. is clearly meant for reading database tables.

A work-around could be to first load your flat file data in a transparent database table (which you have to create using the ABAP Dictionary). This has only to be done once of course (so make sure to check on that). After that you can use OPEN CURSOR etc. from here. Check your data against CRMD_TERR_ACCREL and so on.

Bottom line is that you have to start with your main source data (i.e. the flat file data which you stored in a transparent database table) and use the "fetch data" approach for this table.


Best regards,

Sander



john_lang
Active Participant
0 Kudos

Hi Angavai,

Sanders proposed approach also has the advantage of being able to handle very large flat files and not hit any memory limit issues because the OPEN CURSOR will only pull the next single DataPacket worth of records from the database table.

And remember to add a bit of housekeeping code as well. Where you CLOSE CURSOR also consider adding a statement to delete the data contents out of the database table that was used for the temporary storage of the flat file.

Hope this helps.

Kind regards,

John.

fcorodriguezl
Contributor
0 Kudos

Hi Angavai,

Sounds like missing instructions before your endloop.

IF I_INITFLAG = SBIWA_C_FLAG_ON.

....

  ENDLOOP.

  IF sy-subrc EQ 0.

      CLOSE CURSOR.

      RAISE NO_MORE_DATA.

  ENDIF.

  S_COUNTER = S_COUNTER + 1.

ENDIF.

ENDFUNCTION.

Please share your code. Regards.

sander_vanwilligen
Active Contributor
0 Kudos

Hi Angavai,

Can you please share your coding?

Thanks,

Sander