cancel
Showing results for 
Search instead for 
Did you mean: 

Full load for EM

Former Member
0 Kudos

Does sap provide any function to extract data to BW for full load - for all the EH data? What I see there is delta load. The delta is in table SCEM_EH_SETUP ......

I have EM data in EM system. BW was not configured. Now we configured BW. How to capture all the data without write custom extractor for full load?

Thanks

JC

Accepted Solutions (1)

Accepted Solutions (1)

former_member190756
Active Contributor
0 Kudos

Hello JC,

you can do an initial upload for already existing Event Handlers and later switch to delta upload. It is described in the IMG Activity

Event Management -> SAP Business Information Warehouse Interface ->Process for Extracting SAP BW Data.

The report /SAPTRX/R_BW_EXTRACTION_INIT for that looks similar to transaction /SAPTRX/EH_LIST

The initial upload should happen in an update free time in order to catch all existing Event Handlers.

Best regards,

Steffen

Former Member
0 Kudos

Thanks.

1. I ran the rsa3, I got all my records.

2. I ran transaction /SAPTRX/BWID - delete, then transaction /SAPTRX/BWIU - ini... - followed the instruction.

In RSA3, it returns nothing.

3. I created new data in source system. I can get the new data.

4. I run the delete again, then ini. then rsa3, the new data gone.

How can I get all the data back for BW?

former_member190756
Active Contributor
0 Kudos

Did you change the parameter in the BW activity in your rule set to initial upload after the deletion?

BW_UPLOAD Upload Business Information Warehouse

/SAPTRX/BW_EXTRACTION2

Acty Par. Usage Header Text Parameter Description

Update Type Space = Initial Upload; X = Delta Upload

Best regards,

Steffen

Former Member
0 Kudos

Thanks so much for your reply.

It sets to blank. rsa3 - return nothing.

Former Member
0 Kudos

Hi Steffen,

In spro:

BW_UPLOAD is pointed to /SAPTRX/BW_EXTRACTION2

Acty Par. Usage Header Text Parameter Description

Update Type Space

There are more than 60 records in event handler list.

1. I ran the program /SAPTRX/R_BW_EXTRACTION_INIT with event code 0ini and event id : BWUPLOAD

"Initial upload started for 137 event handler"

nothing return in rsa3 with 'F' for update mode. Extracted from BW with ini with data transfer, 0 records return.

Former Member
0 Kudos

What is the transaction /SAPTRX/BWID doing? In the installation guide, the step is before /SAPTRX/BWIU - ini. But if you run the transaction /SAPTRX/BWID, on the tab, it says "Deletion of setup tables after initial upload".

former_member190756
Active Contributor
0 Kudos

Hello JC,

do all EH have the same rule sets and in all rule sets initial upload is set up?

If you run initial upload /SAPTRX/R_BW_EXTRACTION_INIT all selected EH should get an event BW_UPLOAD. That should be visible if you display the EH in /SAPTRX/EH_LIST. If not maybe the EH was blocked.

Afterwards the stup tables should be filled and you should be able to check the data in RSA3. With /SAPTRX/BWID you delete these setup tables.

Delta upload writes in a delta queue.

Best regards,

Steffen

Former Member
0 Kudos

I ran /SAPTRX/BWID, deleted the ini setup tab before the ini load. Is there a way re-do the ini and have those setup table filled?

former_member190756
Active Contributor
0 Kudos

If you now switch the activity to initial load and run then the report it should work. The setup tables should then be filled.

Former Member
0 Kudos

Hi Steffen,

I think the problem the ini program or maybe some configuration. Ini program runs successful. But there is error in slg1. "Message processing aborted. Not in list of allowed unexp. event codes" . Setup table is not filled.

If I create new record, all these data will fill base on setting. If it is ini, it is in setup table, If it is delta, data is in delta queue.

I cannot get the setup program( /SAPTRX/R_BW_EXTRACTION_INIT) work. Any idea?

Regards

JC

former_member190756
Active Contributor
0 Kudos

Ok, thats the missing hint:

"Message processing aborted. Not in list of allowed unexp. event codes".

You must enter the BWUPLOAD event code in transaction /SAPTRX/TSC0TT as allowed unexpected event code for your Event Handler Type or remove the check for allowed unexpected event codes completely.

Best regards,

Steffen

Former Member
0 Kudos

Hi,

Just to understand.

If I use the 'initial upload' option, then I am writing the extraction data to the following tables:

SCEM_EE_SETUP

SCEM_EG_SETUP

SCEM_EH_SETUP

If I use the 'delta' option, then I don't write to those tables, but write the extraction data to the 0SCEM_1/2/3 queues.  If so, how can I check the data that is being extracted?

Also, if I have an event handler, with 20 events and then I get 1 more event.  Is the expectation that the standard function (assuming delta extraction), will just extract the last event, or will it extract all 21 events?

Thanks for your help!

JP

former_member190756
Active Contributor
0 Kudos

Hello JP,

yes i think it will extract all 21. You can see this in function module /SAPTRX/BW_DATA_PREP2.

For the reason i would have to ask a colleague.

Couldn't you use RSA3 to check the data extraction?

Best regards,

Steffen

Former Member
0 Kudos

Thanks for the quick reply!

I can check for the initial load in RSA3, but not for the delta load.  Do you know how to check the data that was extracted for the delta load (my understanding is that this goes via qRFC into BW, so I could check in SMQ1 (outbound queue)), but can not see the data.

I am confused then on delta   If its a delta extract, doesn't that mean that it will extract the 'delta' and not all the events again (as the previous events did not change).

If you could check, that would be great.

Thanks.

JP

Former Member
0 Kudos

1. to start a fresh initialization, delete the data in the Setup Tables; use t-code (/n/SAPTRX/BWID).

you should see 0 records in rsa3

2.

a.      Set BW upload rules flag (blank space ‘  ‘ for initial .), 

b.     Start the initial load of specific EH data to the Setup Tables with t-code (/n/SAPTRX/BWIU).

3. In EM set rule set for BW_upload

4.  Start t-code (/n/SAPTRX/BWIU)

5. Run rsa3, you should see record.

6. set rule flag for Delta and ‘X’ for delta

7. create data. you should see rsa3.

Former Member
0 Kudos

Hi,

Thanks for the reply.

My understanding, is that the following tables are only filled during the initial load:

SCEM_EE_SETUP

SCEM_EG_SETUP

SCEM_EH_SETUP

If you see the FM - /SAPTRX/BW_UPDATE_V1, it looks like its only updated during the 'initial load'. (but could be my misunderstanding)

Thanks.

JP

Former Member
0 Kudos

Thanks for all of you help. 

I wanted to get the advice from any one who has done an initial load of millions of records.  We have a production database that has around 13 million event handlers and around 130 million events for these handlers.  Would you recommend using the standard SAP initial upload program, or create one to bypass the 'event creation' and just fill the data extraction structures.

If anyone has done this conversion before for this amount of data, please let me know.

Thanks.

JP

former_member190756
Active Contributor
0 Kudos

Hello,

i could now clarify the question:

"Also, if I have an event handler, with 20 events and then I get 1 more event.  Is the expectation that the standard function (assuming delta extraction), will just extract the last event, or will it extract all 21 events?"

The initial design for this activity was that an BW Upload is done when the process that the EH covers ended.

i.e. the process is completed and you only do one BW Upload.

If you have multiple Uploads e.g. every time an Event Message is processed you should think about to copy the activity and adjust it to your needs.

Best regards,

Steffen

Former Member
0 Kudos

Hi,

Thanks for your input.  That is what we did.  We changed the ruleset extraction FM to only select the latest event (during delta mode).  Surprising that standard SAP extracts all events even during delta mode.

Do you have any input on converting a large amount of data.  See below:

I wanted to get the advice from any one who has done an initial load of millions of records.  We have a production database that has around 13 million event handlers and around 130 million events for these handlers.  Would you recommend using the standard SAP initial upload program, or create one to bypass the 'event creation' and just fill the data extraction structures.

If anyone has done this conversion before for this amount of data, please let me know.

Thanks.

JP

former_member190756
Active Contributor
0 Kudos

Important is that you do the initial load in an update free time i.e. no EH were created or changed while doing it.

13 Million EH sounds not so much. So i would recommend to use the standard initial upload.

I'm not sure if the BW Upload can be parallelized. If this would be possible it should be possible to do it in half a day on a respective hardware.

Best regards,

Steffen

Former Member
0 Kudos

Hi,

Thanks for your input.  Even though the standard upload uses the 'old' ADDEVENT BAPI, you don't think we are missing any major functionality?

Also, I can understand running the conversion during a non-update window.

My understanding is that even if I have updates happening, these will write to the DELTA queues.  So we can pull the initial data and then the delta data after that?

Would you agree.

Thanks.

JP

former_member190756
Active Contributor
0 Kudos

My understanding is that even if I have updates happening, these will write to the DELTA queues.  So we can pull the initial data and then the delta data after that?

Hello JP,

no this won't work as you have to set the Activity parameter for the BW Upload Activity either to Initial or Delta. You can check this in the docu for the BW Upload in the IMG.

Thanks for your input.  Even though the standard upload uses the 'old' ADDEVENT BAPI, you don't think we are missing any major functionality?

As during the initial load only a dummy message is sent to trigger the BW upload activity there is no function loss here.

Best regards,

Steffen

Former Member
0 Kudos

Hi,

In our ruleset we have configured it such that:

- If the event is BWUPLOAD - then the activity parameter is blank (initial)

- For all other events - then the activity parameter is X (delta)

We also plan to split the BWIU program by month and run each month in parallel.

Let me know what you think.

Thanks.

JP

former_member190756
Active Contributor
0 Kudos

Discussed it with a colleague.

You can do it in the described way. It is only important that the initial load is done before you do delta load for a record otherwise you can loose data.

Parallelization is possible so you can split it up per month.

Best regards,

Steffen

Former Member
0 Kudos

Hi,

Thanks again for your information!

Could you give me more details why their could be a 'data loss'.  In our case, we will always overwrite in BW (based on the converted GUID being the key). 

For example:

- Delta run - extracts GUID 123

- Initial run - also extracts GUID 123, it will overwrite the entry in BW

Our assumption, is that an event can NEVER change, it will always be a create.

Thanks for looking into this.

JP

former_member190756
Active Contributor
0 Kudos

For the event messages i see no problem. But you should look at the EH data if there is everything working.

As no one has done it in this way you should test this special case.

Best regards,
Steffen

Former Member
0 Kudos

Hi,

Thanks again for your input!

For the delta or initial, we always send the latest EH data to BW, so not sure how this could create any issue.  Am I missing something.

Thanks again!

JP

former_member190756
Active Contributor
0 Kudos

If you can guarantee that during the initial load of the EH no update via Event Message for the same EH is done this should work as described.

Best regards,

Steffen

Answers (0)