on 04-06-2011 12:04 AM
Does sap provide any function to extract data to BW for full load - for all the EH data? What I see there is delta load. The delta is in table SCEM_EH_SETUP ......
I have EM data in EM system. BW was not configured. Now we configured BW. How to capture all the data without write custom extractor for full load?
Thanks
JC
Hello JC,
you can do an initial upload for already existing Event Handlers and later switch to delta upload. It is described in the IMG Activity
Event Management -> SAP Business Information Warehouse Interface ->Process for Extracting SAP BW Data.
The report /SAPTRX/R_BW_EXTRACTION_INIT for that looks similar to transaction /SAPTRX/EH_LIST
The initial upload should happen in an update free time in order to catch all existing Event Handlers.
Best regards,
Steffen
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Thanks.
1. I ran the rsa3, I got all my records.
2. I ran transaction /SAPTRX/BWID - delete, then transaction /SAPTRX/BWIU - ini... - followed the instruction.
In RSA3, it returns nothing.
3. I created new data in source system. I can get the new data.
4. I run the delete again, then ini. then rsa3, the new data gone.
How can I get all the data back for BW?
Did you change the parameter in the BW activity in your rule set to initial upload after the deletion?
BW_UPLOAD Upload Business Information Warehouse
/SAPTRX/BW_EXTRACTION2
Acty Par. Usage Header Text Parameter Description
Update Type Space = Initial Upload; X = Delta Upload
Best regards,
Steffen
Hi Steffen,
In spro:
BW_UPLOAD is pointed to /SAPTRX/BW_EXTRACTION2
Acty Par. Usage Header Text Parameter Description
Update Type Space
There are more than 60 records in event handler list.
1. I ran the program /SAPTRX/R_BW_EXTRACTION_INIT with event code 0ini and event id : BWUPLOAD
"Initial upload started for 137 event handler"
nothing return in rsa3 with 'F' for update mode. Extracted from BW with ini with data transfer, 0 records return.
Hello JC,
do all EH have the same rule sets and in all rule sets initial upload is set up?
If you run initial upload /SAPTRX/R_BW_EXTRACTION_INIT all selected EH should get an event BW_UPLOAD. That should be visible if you display the EH in /SAPTRX/EH_LIST. If not maybe the EH was blocked.
Afterwards the stup tables should be filled and you should be able to check the data in RSA3. With /SAPTRX/BWID you delete these setup tables.
Delta upload writes in a delta queue.
Best regards,
Steffen
Hi Steffen,
I think the problem the ini program or maybe some configuration. Ini program runs successful. But there is error in slg1. "Message processing aborted. Not in list of allowed unexp. event codes" . Setup table is not filled.
If I create new record, all these data will fill base on setting. If it is ini, it is in setup table, If it is delta, data is in delta queue.
I cannot get the setup program( /SAPTRX/R_BW_EXTRACTION_INIT) work. Any idea?
Regards
JC
Ok, thats the missing hint:
"Message processing aborted. Not in list of allowed unexp. event codes".
You must enter the BWUPLOAD event code in transaction /SAPTRX/TSC0TT as allowed unexpected event code for your Event Handler Type or remove the check for allowed unexpected event codes completely.
Best regards,
Steffen
Hi,
Just to understand.
If I use the 'initial upload' option, then I am writing the extraction data to the following tables:
SCEM_EE_SETUP
SCEM_EG_SETUP
SCEM_EH_SETUP
If I use the 'delta' option, then I don't write to those tables, but write the extraction data to the 0SCEM_1/2/3 queues. If so, how can I check the data that is being extracted?
Also, if I have an event handler, with 20 events and then I get 1 more event. Is the expectation that the standard function (assuming delta extraction), will just extract the last event, or will it extract all 21 events?
Thanks for your help!
JP
Thanks for the quick reply!
I can check for the initial load in RSA3, but not for the delta load. Do you know how to check the data that was extracted for the delta load (my understanding is that this goes via qRFC into BW, so I could check in SMQ1 (outbound queue)), but can not see the data.
I am confused then on delta If its a delta extract, doesn't that mean that it will extract the 'delta' and not all the events again (as the previous events did not change).
If you could check, that would be great.
Thanks.
JP
1. to start a fresh initialization, delete the data in the Setup Tables; use t-code (/n/SAPTRX/BWID).
you should see 0 records in rsa3
2.
a. Set BW upload rules flag (blank space ‘ ‘ for initial .),
b. Start the initial load of specific EH data to the Setup Tables with t-code (/n/SAPTRX/BWIU).
3. In EM set rule set for BW_upload
4. Start t-code (/n/SAPTRX/BWIU)
5. Run rsa3, you should see record.
6. set rule flag for Delta and ‘X’ for delta
7. create data. you should see rsa3.
Hi,
Thanks for the reply.
My understanding, is that the following tables are only filled during the initial load:
SCEM_EE_SETUP
SCEM_EG_SETUP
SCEM_EH_SETUP
If you see the FM - /SAPTRX/BW_UPDATE_V1, it looks like its only updated during the 'initial load'. (but could be my misunderstanding)
Thanks.
JP
Thanks for all of you help.
I wanted to get the advice from any one who has done an initial load of millions of records. We have a production database that has around 13 million event handlers and around 130 million events for these handlers. Would you recommend using the standard SAP initial upload program, or create one to bypass the 'event creation' and just fill the data extraction structures.
If anyone has done this conversion before for this amount of data, please let me know.
Thanks.
JP
Hello,
i could now clarify the question:
"Also, if I have an event handler, with 20 events and then I get 1 more event. Is the expectation that the standard function (assuming delta extraction), will just extract the last event, or will it extract all 21 events?"
The initial design for this activity was that an BW Upload is done when the process that the EH covers ended.
i.e. the process is completed and you only do one BW Upload.
If you have multiple Uploads e.g. every time an Event Message is processed you should think about to copy the activity and adjust it to your needs.
Best regards,
Steffen
Hi,
Thanks for your input. That is what we did. We changed the ruleset extraction FM to only select the latest event (during delta mode). Surprising that standard SAP extracts all events even during delta mode.
Do you have any input on converting a large amount of data. See below:
I wanted to get the advice from any one who has done an initial load of millions of records. We have a production database that has around 13 million event handlers and around 130 million events for these handlers. Would you recommend using the standard SAP initial upload program, or create one to bypass the 'event creation' and just fill the data extraction structures.
If anyone has done this conversion before for this amount of data, please let me know.
Thanks.
JP
Important is that you do the initial load in an update free time i.e. no EH were created or changed while doing it.
13 Million EH sounds not so much. So i would recommend to use the standard initial upload.
I'm not sure if the BW Upload can be parallelized. If this would be possible it should be possible to do it in half a day on a respective hardware.
Best regards,
Steffen
Hi,
Thanks for your input. Even though the standard upload uses the 'old' ADDEVENT BAPI, you don't think we are missing any major functionality?
Also, I can understand running the conversion during a non-update window.
My understanding is that even if I have updates happening, these will write to the DELTA queues. So we can pull the initial data and then the delta data after that?
Would you agree.
Thanks.
JP
My understanding is that even if I have updates happening, these will write to the DELTA queues. So we can pull the initial data and then the delta data after that?
Hello JP,
no this won't work as you have to set the Activity parameter for the BW Upload Activity either to Initial or Delta. You can check this in the docu for the BW Upload in the IMG.
Thanks for your input. Even though the standard upload uses the 'old' ADDEVENT BAPI, you don't think we are missing any major functionality?
As during the initial load only a dummy message is sent to trigger the BW upload activity there is no function loss here.
Best regards,
Steffen
Hi,
In our ruleset we have configured it such that:
- If the event is BWUPLOAD - then the activity parameter is blank (initial)
- For all other events - then the activity parameter is X (delta)
We also plan to split the BWIU program by month and run each month in parallel.
Let me know what you think.
Thanks.
JP
Hi,
Thanks again for your information!
Could you give me more details why their could be a 'data loss'. In our case, we will always overwrite in BW (based on the converted GUID being the key).
For example:
- Delta run - extracts GUID 123
- Initial run - also extracts GUID 123, it will overwrite the entry in BW
Our assumption, is that an event can NEVER change, it will always be a create.
Thanks for looking into this.
JP
User | Count |
---|---|
15 | |
4 | |
3 | |
2 | |
1 | |
1 | |
1 | |
1 | |
1 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.