MDM Import manager data sources

[Former Member]

Hi experts,

As far as I know MDM has ETL capabilities with MDM Import/syndicator tools that allows to load and extract data to and from MDM using files and relational databases as source/target.

What different kind of data sources supports MDM Import manager?

When should we work with SAP XI/PI?

When sould we work with ETL (Data Services as Data Integrator)?

Thanks in advance,

Abby

Edited by: Abigail Beurmann on Nov 23, 2011 11:38 AM

[Former Member]

Hello,

What different kind of data sources supports MDM Import manager?

There are number of data sources available from which you can connect your MDM, it can be MS Access, Oracle DB, Excel file, CSV File, MDM Ports, just to name a few, for complete list yo can refer the MDM Import Manager Guide, which is available for download from http://service.sap.com/installmdm71

When should we work with SAP XI/PI?

*The main criteria that decides to use PI is ease the development efforts, as the standard pre-delivered content like MDM Adapter, from MDM 7.1, will convert the IDoc given by ECC system ti XML format and place it in the ready folder,

Also SAP PI being a middleware technology can connect to the Data Service via Web services of MDM.

When sould we work with ETL (Data Services as Data Integrator)?

The depends on the business requirements, if the functionaly provided by MDM doesnt not fullfill for your requirement, then we use the DS, for exmaple if we need to check de-duplicate(more robust in DS than MDM).

Hope this helps you understand a bit better.

regards,

Abhishek

[Former Member]

Thank you very much Abishhek,

I have just checked de source list: Access, Delimited Text, Excel, Fixed Text, Oracle, Port, SQL Server, XML, XML Schema.

So, Could you please tell me if it's a common practice to import data in MDM using files? for example as you said, We want to import data from R/3 to MDM using XML files that XI/PI generates?

In our business scenario, We have a SAP CRM with SQL server 2005 as a source. Could MDM automatically import data directly from SQL Server or it's a better practice to generate XML files with PI using the Import Port/folder?

Thanks again,

Abby

Edited by: Abigail Beurmann on Nov 23, 2011 1:29 PM

[Former Member]

Hi Abigail,

So, Could you please tell me if it's a common practice to import data in MDM using files?

for example as you said, We want to import data from R/3 to MDM using XML files that XI/PI generates?

Yes its a common practice to import data in MDM in files.Most common mode of data exchange is xml which has many advantages namely it is accepted open standard,allows to clearly separate content from form (appearance), is text-oriented,is Secure, is extensible,is self-describing, is universal; meaning internationalization is no problem and XML is a future-oriented technology.

In our business scenario, We have a SAP CRM with SQL server 2005 as a source. Could MDM automatically import data directly from SQL Server or it's a better practice to generate XML files with PI using the Import Port/folder?

One should always use a interface as business rules can possibly change and making a later change could be huge ask.

So PI can act a interface between MDM and CRM.

CRM supports Bdoc and PI can integrate with CRM with help of ABAP Proxies.

The following thread provides you in depth information on integration of the 2 systems:

To help you understand more about webservice and PI integration with MDM for CRM integration please go though this thread:

To know more about PI vs DS please go though this guide:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/10fbac70-c381-2d10-afbe-c3902a694...

About Integration with PI:

http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/8035bf61-d6b5-2e10-5fa6-e5e00b9e6174&override...

A guide which takes you through the entire process of CRM PI MDM integration,although for SAP MDM 5.5 version but valid for most part:

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/405a73cc-9068-2c10-859e-c13f81922...

Hope this helps.

Thanks,

Ravi

[Former Member]

Thank you very much for all your answers,

Just a few more questions about The depends on the business requirements, if the functionaly provided by MDM does not fullfill for your requirement, then we use the DS, for example if we need to check de-duplicate(more robust in DS than MDM). sentence that justifies Data Services usage.

As far as I know, When we work with MDM, merge strategies (Score value) are just based on field tuples. When you say that de-duplicate is more robust in DS than MDM. Does it mean that DS includes algorithms that helps to obtain a more "elaborated" Score value? Could you kindy give me an example based on your experience?

I have read more documentation related to the Data Services capabilites that fulfills an scenario requirement that I'd like to clarify:

- Enriching the set of available transformations than can be used in the process: As far as I have read Data Services (Data quality) includes some templates that allows to validate/transform some registers, for instance a client address template. Does it mean that DS detects wrong adresses or just correct the adress format? Could you please refer me to other transformations that Data Services includes in a Master Data scenario (Client master data) for example "Name validation"? Do you know if it is possible just to import and benefit of these templates in MDM workflow instead of transform data before importing it in MDM?

- enable an incremental update (a Data services Job is used to generate an import file than contains only the updated records:

Does it mean that DS match source records (full data) with MDM repository and only load "new" and "delta" records in MDM instead of load all data?

- *merge data automatically": When we work with MDM, Data manager shows a Score that allows user to merge records manually (manual consolidation). Is it possible to merge data and create a golden register automatically using Data Services?

Thanks and regards,

Abby

[Former Member]

Just a few more questions about The depends on the business requirements, if the functionaly provided by MDM does not fullfill for your requirement, then we use the DS, for example if we need to check de-duplicate(more robust in DS than MDM). sentence that justifies Data Services usage.

As far as I know, When we work with MDM, merge strategies (Score value) are just based on field tuples. When you say that de-duplicate is more robust in DS than MDM. Does it mean that DS includes algorithms that helps to obtain a more "elaborated" Score value? Could you kindy give me an example based on your experience?

The alogorithm used by DS is more robust and this usually happends on the fly, when the records is created, at that time only, one can trrigger using the workflow and avoid duplicated gettiing created in the MDM system.

I have read more documentation related to the Data Services capabilites that fulfills an scenario requirement that I'd like to clarify:

- Enriching the set of available transformations than can be used in the process: As far as I have read Data Services (Data quality) includes some templates that allows to validate/transform some registers, for instance a client address template. Does it mean that DS detects wrong adresses or just correct the adress format? Could you please refer me to other transformations that Data Services includes in a Master Data scenario (Client master data) for example "Name validation"? Do you know if it is possible just to import and benefit of these templates in MDM workflow instead of transform data before importing it in MDM?

DS has the capability to correct the address format

for documents you can refer to help.sap.com from here you can navigate to the business objects site.

- enable an incremental update (a Data services Job is used to generate an import file than contains only the updated records:

Does it mean that DS match source records (full data) with MDM repository and only load "new" and "delta" records in MDM instead of load all data?

DS will match full data in the MDM repository

- *merge data automatically": When we work with MDM, Data manager shows a Score that allows user to merge records manually (manual consolidation). Is it possible to merge data and create a golden register automatically using Data Services?

Merge is possible but it is not recommended

regards,

Abhishek

Edited by: Abhishek Biwal on Nov 24, 2011 2:25 PM

[Former Member]

Hey Abby,

This thread provides answer to most of your queries pertaining to DS:

This guide give you an example to data service for enhancing data quality :

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/20cf63ef-0d2d-2d10-df8e-d2ef1c848...

Merge Automatically is highly unsafe practice and is not advisable.

Thanks,

Ravi

[Former Member]

Thank you! A very helpful answer!

Please I just need the last question in order to understand the hole MDM+BO scenario.

Imagine that a master data record has been created 3 times in the same data source. Using BO+MDM capabilities we detect the duplicate data and consolidate this data in MDM Data manager (golden register). When syndicating it back to the source with syndicator manager What is going to happen? The 2 extra records will be removed from source? How is the best way to face this situation? Should we first consolidate each data source before consolitate several data sources?

Thanks and regards,

Abby.

[Former Member]

Hello,

I assume by consolidate you mean, you have 3 duplicate data records based on some matching strategy that you have applied or you've got 3 records from different source systems. please clarify

But while importing you must have selected a matching criteria for uniqueness.

Based on this, you will have a fields or combination of fields which identify the records uniquely.

For exmaple, material number might the unique fields. based on this you can select the material number while syndicating the record.

so only one record will get syndicated at a time.

Unless you merge the 3 duplicate ecords into a single records. every records can be identified uniquely, this does not necessarly mean it is a unique records.

If you can please provide an example of data set as duplicates

regards,

Abhishek

[Former Member]

Hey Abby,

I guess what you expect here is how are compacted records handled in target system once compaction done in MDM/MDM-BODS.

Once data compaction is done in MDM it assigns the same remote key values to the multiple compacted records.

So although multiple records in MDM mean the same in MDM,they can mean multiple different in the source system.

Hence any reporting done on MDM will be a precise one.You can also flag the records for deletion in source system after confirming appropriate transactional processes are not running.

Also,I assume you are referring to Data duplication process,should that be one system at a time or multiple at once.

One system at a time is a better approach as set of rules prevail more widely to the data available in one system and hence results are more manageable and accurate.So incremental approach is better.

And to acertain that data quality is high one has to set up governance model and also ensure import is done on the basis of appropriate matching fields.

Hope this helps.

Thanks,

Ravi