SPM datasources - delta extraction not working

[Former Member]

Hi,

We have implemented SPM extractors using the option to generate flat files. We want now change to the option with datasources, but we are encountering several issues:

1) Using the option 'Generate datasource' in Z_SA_DEPD transaction: several values are hardcoded (all are transactional datasources - even the ones for master data -, texts / descriptions are left blank and cannot be changed after that). But most important deltas are not working.

We completed all steps: datasource generated, table OPM_SOURCES configured, infopackages created. When I run it, it is always a full extraction.

2) If I try to generate a generic delta datasource (RSO2 transaction) with the SAP generated function module and extraction structure for the extractor, I receive an error (parameter table missing) that prevents me from saving the datasource.

Any help how can I make a delta extraction work?.

Many thanks,

Regards.

[Former Member · Accepted solution]

Hi Claudia,

Delta extraction in SPM datasources is managed by a date range parameter. During the initial load all the data is picked and from subsequent load only the data changed since last extraction is picked. So you dont require a delta queue mechanicsm. Let me know if this answers your question.

Thanks,

Divyesh

[Former Member · Accepted solution]

Again, Claudia's question is related to mine, yes the date range works with the data amangement UI tool but the emphasis here is on using BW datasources, instead of using the flex data management tool.

Hope that helps.

[Former Member · Accepted solution]

Hi Pom

Both (your and Caludia's) of your questions might actually be different.

For your question, if you want to skip data mgmt tool of SPM you can, but the onus of bringing the data into SPM is on you. You could write your own transformations to connect to your own data sources. There is no code from SAP available to make it work for you out of the box. You could potentially use the SPM extractors and manage your own date range for delta purposes.

For Caludia, assuming you are using SPM Data Mgmt tool, hooked up to the SPM extractor starter kit using data sources, the delta (based on date ranges) would be managed automatically by the data mgmt tool.

Regards

Rajesh

[Former Member · Accepted solution]

Thanks for all responses. I could finally make the datasource work in 'delta' mode. The clue was, after creating the infopackage, you need to specify a date range in the infopackage filter for DATE_RANGE. Otherwise, the parameters SIGN = I, OPTION = BT are not populated correctly. Also, the 'To' value must be left blank if we want the system to default '99991231' value.

By the way, I noticed the datasource created with the the option 'Generate Datasource' from Z_SA_DEPD transaction contains some errors (for example, no text / description is provided for the datasource). Text cannot be added later.

So, I'd prefer to create the datasource manually with transaction RSO2, indicating Extraction method F1 (Function) and function module generated by the tool (example Z_SAPA4PRODUCT_DS) and Extract Structure (ZEXTR_PA4PRODUCT). But when trying to save, I receive an error because of a missing parameter table.

Do you know if possible to create the datasources manually with RSO2?

Thanks!

[Former Member · Accepted solution]

Hi Claudia,

Just found out that the way we've been trying to make the generic datasources delta enabled is not the correct way as hte generated spm function modules, etc will not make the generic delta work. The only way to enable the BW datasources to work correctly is yes mimick the behaviour of the SPM flex ui data management tool, thus use full infopackages and change the dates accordingly.

It does mean full loads but by applying code in the infopackages, you should be able to solve this.

Appreciate full loads but should be too bad, assuming you've got a standard dso in place to take account of change logs etc.

My only other thought of getting a generic delta in the source would have been to create a generic datasource from scratch but even that won't work as the SPM objects are all based on multiple table relationships, hence trying to even find a change pointer will be impossible as it means multiple change pointers.

Your way of amending the infopackage is correct.

Best regards,

Pom

[Former Member · Accepted solution]

Hi Pom,

The function modules generated using SPM datasources are geared specifically towards SPM extractors and like you mentioned there are multiple tables involved so would be hard to replicate this logic using generic datasources. So using these function modules in creating your own generic delta enabled datasources would not be recommended (also we have not tried this mechanism to confirm or deny this solution) and hence the recommendation.

The SPM datasources do use dates to maintain and pull delta records. If left untouched the first load can be considered as a full load and if successful the next load would use the last successful date/time (-5 as a factor of safety) as the start pointer and today's date/time as the end pointer. If this load is successful the date/time would be recorded in an internal table and would be used as the used as the start pointer (ofcourse again the factor of safety would be applied to every load).

Hope this explanation helps.

Rohit

[Former Member · Accepted solution]

Hi Rohit,

Yes, exactly the way we have it and were discussing this yesterday too.

Best regards,

Pom

[Former Member · Accepted solution]

Hi Claudia - We are in the exact same situation as yours and are trying to make Delta load mechanism work. Unfortunately, even after following this thread and trying to execute the load from the SPM portal for Cleared invoices after leaving the date range blank, the extractor was pulling the "full load" and not just delta from the last executiond date. You mentioned in your last response something about the sign and the Option - How did you set those? Prompt help will be greatly appreciated.

Thanks - Kaushal

[Former Member · Accepted solution]

Hi Kaushal,

Are you using datasources, not flat files, right?. In that case, in the infopackage selection criteria, specify a range for DATA_RANGE parameter. Populate with any date in 'From' and '99991231' in 'To'. parameter The SPM UI will overwrite with the date period determined by the system for the new delta extraction, but it does not work if you leave the date range is left blank.

Regards,

Claudia.

[Former Member · Accepted solution]

Hi Claudia,

Just wanted to thank you for your quick response and also that your suggestion was correct in order to make the delta work. Sorry for the delay in getting back to you.

Thanks,

Kaushal.