on 10-11-2007 9:29 PM
Hello All,
I am desparately looking for some help from anybody who knows delta functionality for Generic Extractor Function Module.
My Situation is like this..
I have developed a FM by copying "RSAX_BIW_GET_DATA_SIMPLE" initially. It is working fine for full load. Now I have to make it delta load enable. I have been trying to do this way since long.
I made changes to DS in RSO2 as delta enabled on TIMESTAMP field(It is for FAGLFLEXA table basically). It is not being populated into i_t_select list when the function module is being called by SAPI. So I am unable to write any code to pull delta records.
Even I tried with "RSVD_BW_GET_DELTA_DATA" template but here I_ISOURCE and I_UPDMODE fields are being populated into the FM when it is called by SAPI(RSA3).
I am desparate with this FM since a month. Can anybody help me please. I will give points.
I have been searching this forum as well but I could not found any correct solution. Need step by step if possible. I am developing this for FAGLFLEXA table in FI.
Thanks,
Hanuma
Hi Hanuma, I am also having the same requirement to create the FM for FAGLFLEXA, Please send me your ABAP Code to me at soniyakv@hotmail.com
if you can please help me will appreciate your help. Kindly also let me know your steps since I am good in BI so will catch it.
Thanks
Sincerely
Soniya Kapoor
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Hanuma,
You have to take care of the delta in the extractor function module itself. Check the type of the extraction. There could be initial load, full load, test extraction etc etc. Maintain a table, where in which you have the time of extraction of last deltas, the lower limit and the upper limit. In the next extraction, check the upper limit of the last extraction, and add one second to it and then start extracting the data from the FAGLFLEXA table till the current system time. If the extraction type is delta, then enter one more row for lower and upper limit in the ytable. This was we are extractiing COPA data.
Regards
Sriram
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
I have written all the code required.
How do we know type of extraction? I am thinking based on I_UPDMODE field, but it is not coming into function module from SAPI(RSA3 program). I checked it in debugging. Can you please give me your mail id so that I can send you what I did step by step. So that you may understand where I am doing wrong. I am looking for some help desparately.
Thnak You,
Hanuma
Hi,
I think having answered to a similar thread.
If you have created a generic DataSource based on FM extractor with a delta field based on one field of your structure (TIMESTAMP); the delta will work for sure.
If your init your delta is successful, you will see entries in table ROOSGENDLM
in your source system.
When you request a delta, the TIMESTAMP will be passed to the itab I_T_SELECT. You'll have to read it with for instance
READ TABLE I_T_SELECT INTO L_S_SELECT
WITH KEY FIELDNM = 'YOUR_DELTA_FIELD_NAME'.
Then you'll open your cursor
OPEN WITH HOLD G_CURSOR FOR
SELECT FIELDS FROM TABLE
WHERE YOUR_DELTA_FIELD_NAME'>= :L_S_SELECT-LOW
AND YOUR_DELTA_FIELD_NAME <= :L_S_SELECT-HIGH.
...
Now comes the challenges: do you have this 'YOUR_DELTA_FIELD_NAME' available? I mean is the timestamp posted in your TABLE?
If not then it will remain empty and that won't work...
let us know
Olivier.
I did not see any TIMESTAMP field in I_T_SELECT table eventhough I made TIMESTAMP as delta enabled in RSO2 for my DS.
My function module is being called from "RSA3_BIW_GET_DATA_SIMPLE" from SAPI when I see in debugging. It is also not passing I_UPDMODE field to my FM.
My FM is working fine for full load. But there are the two main problems I am facing to get my FM as delta enabled.
Problem#1. field TIMESTAMP is not being populated into I_T_SELECT even I pass update mode as 'F' or 'D'.
for F, I am getting all records.
for D, I am getting error like "Errors occoured in data extraction".
Probelm#2.
I_UPDMODE field is not coming into my FM.
I think you may have done it earlier, please send me if you have some step by step procedure. my mail id is hgujjula@gmail.com
Regards,
Hanuma
The step by step procedure is as follows:
- Create your extract structure (let's name it EXTRSTR); this extract structure will have a field on which your Delta (of type timestamp or other supported); its name: DELTAFLD.
- create a function module (EXTRFM) as a copy of RSAX_BIW_GET_DATA_SIMPLE.
- Create a generic DSource (GENDS) based on EXTRSTR and FM EXTRFM. Enable this generic DS for delta, type timestamp on field DELTAFLD.
You Function module:
FUNCTION EXTRFM.
*"----------------------------------------------------------------------
*"*"Local interface:
*" IMPORTING
*" VALUE(I_REQUNR) TYPE SBIWA_S_INTERFACE-REQUNR
*" VALUE(I_ISOURCE) TYPE SBIWA_S_INTERFACE-ISOURCE OPTIONAL
*" VALUE(I_MAXSIZE) TYPE SBIWA_S_INTERFACE-MAXSIZE OPTIONAL
*" VALUE(I_INITFLAG) TYPE SBIWA_S_INTERFACE-INITFLAG OPTIONAL
*" VALUE(I_UPDMODE) TYPE SBIWA_S_INTERFACE-UPDMODE OPTIONAL
*" VALUE(I_DATAPAKID) TYPE SBIWA_S_INTERFACE-DATAPAKID OPTIONAL
*" VALUE(I_PRIVATE_MODE) OPTIONAL
*" VALUE(I_CALLMODE) LIKE ROARCHD200-CALLMODE OPTIONAL
*" TABLES
*" I_T_SELECT TYPE SBIWA_T_SELECT OPTIONAL
*" I_T_FIELDS TYPE SBIWA_T_FIELDS OPTIONAL
*" E_T_DATA OPTIONAL
*" E_T_EXTRSTR STRUCTURE EXTRSTR
*" OPTIONAL
*" EXCEPTIONS
*" NO_MORE_DATA
*" ERROR_PASSED_TO_MESS_HANDLER
*"----------------------------------------------------------------------
* Auxiliary Selection criteria structure
DATA: L_S_SELECT TYPE SBIWA_S_SELECT.
STATICS: S_S_IF TYPE SRSC_S_IF_SIMPLE,
S_COUNTER_DATAPAKID LIKE SY-TABIX.
IF I_INITFLAG = SBIWA_C_FLAG_ON.
* Invalid second initialization call -> error exit
IF NOT G_FLAG_INTERFACE_INITIALIZED IS INITIAL.
IF 1 = 2. MESSAGE E008(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'008' "message number
' ' "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDIF.
* Check InfoSource validity
CASE I_ISOURCE.
WHEN 'GENDS'.
WHEN OTHERS.
IF 1 = 2. MESSAGE E009(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'009' "message number
I_ISOURCE "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDCASE.
* Check for supported update mode
CASE I_UPDMODE.
WHEN 'F'.
WHEN 'D'.
WHEN 'R'.
WHEN OTHERS.
IF 1 = 2. MESSAGE E011(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'011' "message number
I_UPDMODE "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDCASE.
READ TABLE I_T_SELECT INTO L_S_SELECT
WITH KEY FIELDNM = 'DELTAFLD'.
APPEND LINES OF I_T_SELECT TO G_T_SELECT.
* Fill parameter buffer for data extraction calls
G_S_INTERFACE-REQUNR = I_REQUNR.
G_S_INTERFACE-ISOURCE = I_ISOURCE.
G_S_INTERFACE-MAXSIZE = I_MAXSIZE.
G_S_INTERFACE-INITFLAG = I_INITFLAG.
G_S_INTERFACE-UPDMODE = I_UPDMODE.
G_S_INTERFACE-DATAPAKID = I_DATAPAKID.
G_FLAG_INTERFACE_INITIALIZED = SBIWA_C_FLAG_ON.
ELSE. "Initialization mode or data extraction ?
* First data package -> OPEN CURSOR
CASE G_S_INTERFACE-ISOURCE.
LOOP AT G_T_SELECT INTO L_S_SELECT
WHERE FIELDNM = 'DELTAFLD'.
ENDLOOP.
IF G_COUNTER_DATAPAKID = 0.
IF L_S_SELECT-LOW = '' AND L_S_SELECT-HIGH = ''. "FULL or INIT
OPEN WITH HOLD G_CURSOR FOR
SELECT <FIELDS>
FROM TABLE1, TABLE2.....
ELSE. "DELTA
EXEC SQL.
SELECT <FIELDS>
FROM TABLE1, TABLE2.....
WHERE DELTAFLD >= L_S_SELECT-LOW
AND DELTAFLD <= L_S_SELECT-HIGH
ENDIF.
ENDIF.
DO.
lv_recordcount = 1.
FETCH NEXT G_CURSOR INTO workarea
IF sy-subrc <> 0.
EXIT.
ELSE.
APPEND workarea to E_T_EXTRSTR.
IF lv_recordcount = G_S_INTERFACE-MAXSIZE.
EXIT.
ELSE.
lv_recordcount = lv_recordcount + 1.
ENDIF.
ENDIF.
ENDDO.
IF sy-subrc <> 0.
RAISE NO_MORE_DATA.
lv_recordcount = 1.
ENDIF.
G_COUNTER_DATAPAKID = G_COUNTER_DATAPAKID + 1.
************************************************************************
ENDCASE.
ENDIF.
ENDFUNCTION.
For the problem number#2 you are actuall right; the system doesn't provide the update mode; you'll have as well to mainain the table roosource after having created your DSource:
it should look like:
Field Value
OLTPSOURCE GENDS
OBJVERS A
TYPE TRAN
APPLNM BW
BASOSOURCE
DELTA E
STOCKUPD
UPDFLG1
UPDFLG2
UPDFLG3
READONLY X
INITSIMU 1
INITCOMMIT
ZDD ABLE X
EXMETHOD F1
EXTRACTOR EXTRFM
EXSTRUCT EXTRSTR
GENFLAG
VIRTCUBE 1
APPENDFLAG
TSTPNM OCORA
TSTPDAT 11.12.2003
TSTPTIM 18:10:12
INVFIELD
GENDELTAFD DELTAFLD
GENDELTATP T
DELTASAFE1
DELTASAFE2 600
DLTAGETMAX
SEGMENT
AFTERIMP
BEFOREXP
MAINTAIN
CHECKDS
SAVEDS
TRANSPDS
DELETE DS
RESETDELTA RSA8_GENDELTA_RESET
APPL PROP
ACTIVATE
ZDD REPLAY RSA8_GENDELTA_ZDD_REPLAY_GET
TFMETHODS
....
In particular, pay attention at the extraction method: it should be F1 (complete interface).
hope this helps...
Olivier.
Thanks for your code.
But I have been doing the same for a week. Again I did new set of extract structure, FM and DataSource. But getting the same error.
Please see my code here..
Step 1: Creating Extract Structure in SE11
Field TIMESTAMP is used for Delta.
Step 2: Creating Function Module based on RSAX_BIW_GET_DATA in SE37
I copied the above function module to ZEXTR_FLEXA.
FUNCTION ZEXTR_FLEXA.
*"----
""Local Interface:
*" IMPORTING
*" VALUE(I_REQUNR) TYPE SBIWA_S_INTERFACE-REQUNR
*" VALUE(I_ISOURCE) TYPE SBIWA_S_INTERFACE-ISOURCE OPTIONAL
*" VALUE(I_MAXSIZE) TYPE SBIWA_S_INTERFACE-MAXSIZE OPTIONAL
*" VALUE(I_INITFLAG) TYPE SBIWA_S_INTERFACE-INITFLAG OPTIONAL
*" VALUE(I_UPDMODE) TYPE SBIWA_S_INTERFACE-UPDMODE OPTIONAL
*" VALUE(I_DATAPAKID) TYPE SBIWA_S_INTERFACE-DATAPAKID OPTIONAL
*" VALUE(I_PRIVATE_MODE) OPTIONAL
*" VALUE(I_CALLMODE) LIKE ROARCHD200-CALLMODE OPTIONAL
*" VALUE(I_REMOTE_CALL) TYPE SBIWA_FLAG DEFAULT SBIWA_C_FLAG_OFF
*" TABLES
*" I_T_SELECT TYPE SBIWA_T_SELECT OPTIONAL
*" I_T_FIELDS TYPE SBIWA_T_FIELDS OPTIONAL
*" E_T_DATA OPTIONAL
*" E_T_ZEXTRFLEXA STRUCTURE ZEXTRFLEXA OPTIONAL
*" EXCEPTIONS
*" NO_MORE_DATA
*" ERROR_PASSED_TO_MESS_HANDLER
*"----
TABLES: FAGLFLEXA.
DATA: I_ZEXTRFLEXA LIKE ZEXTRFLEXA OCCURS 0 WITH HEADER LINE.
DATA: LV_RECORDCOUNT TYPE SY-INDEX.
Auxiliary Selection criteria structure
DATA: L_S_SELECT TYPE SBIWA_S_SELECT.
Maximum number of lines for DB table
STATICS: l_maxsize TYPE sbiwa_s_interface-maxsize.
STATICS: S_S_IF TYPE SRSC_S_IF_SIMPLE,
S_COUNTER_DATAPAKID LIKE SY-TABIX.
Select ranges
RANGES: L_R_RBUKRS FOR FAGLFLEXA-RBUKRS,
L_R_DOCNR FOR FAGLFLEXA-DOCNR.
Parameter I_PRIVATE_MODE:
Some applications might want to use this function module for other
purposes as well (e.g. data supply for OLTP reporting tools). If the
processing logic has to be different in this case, use the optional
parameter I_PRIVATE_MODE (not supplied by BIW !) to distinguish
between BIW calls (I_PRIVATE_MODE = SPACE) and other calls
(I_PRIVATE_MODE = X).
If the message handling has to be different as well, define Your own
messaging macro which interprets parameter I_PRIVATE_MODE. When
called by BIW, it should use the LOG_WRITE macro, otherwise do what
You want.
Initialization mode (first call by SAPI) or data transfer mode
(following calls) ?
IF I_INITFLAG = SBIWA_C_FLAG_ON.
************************************************************************
Initialization: check input parameters
buffer input parameters
prepare data selection
************************************************************************
The input parameter I_DATAPAKID is not supported yet !
Invalid second initialization call -> error exit
IF NOT G_FLAG_INTERFACE_INITIALIZED IS INITIAL.
IF 1 = 2. MESSAGE E008(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'008' "message number
' ' "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDIF.
Check InfoSource validity
CASE I_ISOURCE.
WHEN 'ZEXTR_FLEXA'.
WHEN OTHERS.
IF 1 = 2. MESSAGE E009(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'009' "message number
I_ISOURCE "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDCASE.
Check for supported update mode
CASE I_UPDMODE.
WHEN 'F'.
WHEN 'D'.
WHEN 'R'.
WHEN OTHERS.
IF 1 = 2. MESSAGE E011(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'011' "message number
I_UPDMODE "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDCASE.
Check for obligatory selection criteria
READ TABLE I_T_SELECT INTO L_S_SELECT WITH KEY FIELDNM = 'PGMID'.
IF SY-SUBRC <> 0.
IF 1 = 2. MESSAGE E010(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'010' "message number
'PGMID' "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDIF.
READ TABLE I_T_SELECT INTO L_S_SELECT
WITH KEY FIELDNM = 'TIMESTAMP'.
APPEND LINES OF I_T_SELECT TO G_T_SELECT.
Fill parameter buffer for data extraction calls
G_S_INTERFACE-REQUNR = I_REQUNR.
G_S_INTERFACE-ISOURCE = I_ISOURCE.
G_S_INTERFACE-MAXSIZE = I_MAXSIZE.
G_S_INTERFACE-INITFLAG = I_INITFLAG.
G_S_INTERFACE-UPDMODE = I_UPDMODE.
G_S_INTERFACE-DATAPAKID = I_DATAPAKID.
G_FLAG_INTERFACE_INITIALIZED = SBIWA_C_FLAG_ON.
Fill field list table for an optimized select statement
(in case that there is no 1:1 relation between InfoSource fields
and database table fields this may be far from beeing trivial)
APPEND LINES OF i_t_fields TO g_t_segfields.
ELSE. "Initialization mode or data extraction ?
First data package -> OPEN CURSOR
LOOP AT G_T_SELECT INTO L_S_SELECT
WHERE FIELDNM = 'TIMESTAMP'.
ENDLOOP.
IF G_COUNTER_DATAPAKID = 0.
IF L_S_SELECT-LOW = '' AND L_S_SELECT-HIGH = ''. "FULL or INIT
OPEN CURSOR WITH HOLD G_CURSOR FOR
SELECT (G_T_FIELDS)
FROM FAGLFLEXA WHERE RBUKRS IN L_R_RBUKRS
AND DOCNR IN L_R_DOCNR.
ELSE. "DELTA
OPEN CURSOR WITH HOLD G_CURSOR FOR
SELECT (G_T_FIELDS) FROM FAGLFLEXA
WHERE RBUKRS IN L_R_RBUKRS
AND DOCNR IN L_R_DOCNR
AND TIMESTAMP >= L_S_SELECT-LOW
AND TIMESTAMP <= L_S_SELECT-HIGH.
ENDIF.
ENDIF.
FETCH NEXT CURSOR G_CURSOR
APPENDING CORRESPONDING FIELDS
OF TABLE E_T_ZEXTRFLEXA
PACKAGE SIZE G_S_INTERFACE-MAXSIZE.
IF SY-SUBRC <> 0.
CLOSE CURSOR G_CURSOR.
RAISE NO_MORE_DATA.
ENDIF.
G_COUNTER_DATAPAKID = G_COUNTER_DATAPAKID + 1.
ENDIF. "Initialization mode or data extraction ?
ENDFUNCTION.
Step 3: Creating DataSource in RSO2
Step 3: Testing the data load using RSA3
After I modified the value of Extraction Method to F1 in ROOSOURCE table, I am getting same error.
I think screen shots are missing here. Please give me your email address, I will send u the step by step document I did today.
Hanuma
Hi,
can you tell more about the error message?
otherwise, I would start by simplifying and make it work
please comment this part for the moment:
* Check for obligatory selection criteria
READ TABLE I_T_SELECT INTO L_S_SELECT WITH KEY FIELDNM = 'PGMID'.
IF SY-SUBRC <> 0.
IF 1 = 2. MESSAGE E010(R3). ENDIF.
LOG_WRITE 'E' "message type
'R3' "message class
'010' "message number
'PGMID' "message variable 1
' '. "message variable 2
RAISE ERROR_PASSED_TO_MESS_HANDLER.
ENDIF.
Remove your ranges as well (you'll reactivate them after but I am not sure if deltas with selections are supported...)
IF L_S_SELECT-LOW = '' AND L_S_SELECT-HIGH = ''. "FULL or INIT
OPEN CURSOR WITH HOLD G_CURSOR FOR
SELECT (G_T_FIELDS)
FROM FAGLFLEXA .
* WHERE RBUKRS IN L_R_RBUKRS
* AND DOCNR IN L_R_DOCNR.
ELSE. "DELTA
OPEN CURSOR WITH HOLD G_CURSOR FOR
SELECT (G_T_FIELDS) FROM FAGLFLEXA
WHERE TIMESTAMP >= L_S_SELECT-LOW
AND TIMESTAMP <= L_S_SELECT-HIGH.
* RBUKRS IN L_R_RBUKRS
* AND DOCNR IN L_R_DOCNR
ENDIF.
Then I would replace
FETCH NEXT CURSOR G_CURSOR
APPENDING CORRESPONDING FIELDS
OF TABLE E_T_ZEXTRFLEXA
PACKAGE SIZE G_S_INTERFACE-MAXSIZE.
IF SY-SUBRC <> 0.
CLOSE CURSOR G_CURSOR.
RAISE NO_MORE_DATA.
ENDIF.
G_COUNTER_DATAPAKID = G_COUNTER_DATAPAKID + 1.
by
"1 add the declaration:
E_S_ZEXTRFLEXA_PY LIKE E_T_ZEXTRFLEXA.
"right after the LOOP for TIMESTAMP.
"the FETCH
LV_RECORDCOUNT = 1. "control the MAXSIZE
DO.
FETCH NEXT G_CURSOR INTO E_S_ZEXTRFLEXA. "try fetching into a structure
IF sy-subrc <> 0.
EXIT.
ELSE.
APPEND E_S_ZEXTRFLEXA to E_T_ZEXTRFLEXA. "and append the work area in the datapacket table
IF LV_RECORDCOUNT = G_S_INTERFACE-MAXSIZE.
EXIT.
ELSE.
LV_RECORDCOUNT = LV_RECORDCOUNT + 1.
ENDIF.
ENDIF.
ENDDO.
IF sy-subrc <> 0.
IF LV_RECORDCOUNT = 1.
RAISE NO_MORE_DATA.
ENDIF.
LV_RECORDCOUNT = 1.
ENDIF.
G_COUNTER_DATAPAKID = G_COUNTER_DATAPAKID + 1.
In ROOSOURCE I would start first with
Delta Process = 'E'
Safety Time1: empty
Safety time2: some minutes minimum
let us know...
Olivier.
Actually I am ABAP Consultant and new to this BI environment and trying my level best to pull the data as my BW guys wanted.
I guess we can remove all the selection criteria. I want it to be worked first, Then I will make the changes as they want. I will try as you suggested and let you know soon.
Thanks for your help my friend.
Hanuma
Hi,
I created a DataSource in RSO2 successfully with Delta field enable on TIMESTAMP using Function Module and Extract structure.
It is taking extraction method as 'F2" by default so I changed it to 'F1' with my program. This is all done on ROOSOURCE table. I am sure that I changed the only one field which is needed.
Then if I open my DataSource in RSO2, Function module and Extract Structure fields were empty.
Can anybody let me know how to get the fields back with values in it?
Hanuma
Hi Hanuma,
how is it going with this?
I told you: as soon as you change the datasource definition parameters directly in ROOSOURCE, you won't be able to manage it with RSO2; if you want to get it back with the standard RSO2 transaction, the the is to delete the datasource and to recreate it.
In adition please make exactly the settings I suggested, in particular the delta = 'E' in ROOSOURCE.
The first goal is to make it work without any enhancement then we'll advise.
I also wanted to know how is your timestamp fiedl defined and what kind of values it is populated with...
Olivier.
Hello Olivier,
I appreciate your message on FM based delta extractor subject.
One question that I have is about ODS compatibility. Could I update data to ODS using a FM based delta extractor?
"Delta process" setting you recommend is "E". The description of this method is Unspecific delta via extractor (Not ODS compatible).
Thanks in advance,
June Kwon
Global Supply Chain IT
Hewlett Packard Company
User | Count |
---|---|
80 | |
9 | |
9 | |
7 | |
7 | |
6 | |
6 | |
6 | |
5 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.