MDM performance

[Former Member]

Hello

We are in a blueprint phase of a MDM project. We want to use MDM for products and price conditions. We have about 100' products and about 4 millions price conditions. We plan to create a repository where the main table is the priceconditions (4 mill records) and using a lookup table for the materials (about 100' records). We receive updates on both materials and price conditions daily.

I have two questions here:

Does anyone have any experiance with such large amount of data (performence and so on and so forth)?

And (a specific one). The price conditions file that we receive contains several materials that we don't have. Is it possible to NOT import these records (records that don't automap to the lookup table materials)

Regards,

Anders

[Former Member]

Hello Anders,

With such bulk data ,the performance of MDM will surely go down esp. of Syndicator and Data manager.You can follow some steps like delete the system tables which are not of use.Use least no of display fields. and also search parameters,Qualified and taxonomy tables.You can find a document regarding this on sdn.You can also break the source files and then import them.

To neglect the unwanted fields ,u can avoid their mapping

but the values under "Value Mapping " tab had to add without which import can be done you will be prompt for that i.e Values are not mapped.If u want u can add those values in single step by selecting themm all and in right hand side click add.

After importing u can delete these values manually or mapp them to "NULL" or try to change the source file structure as (in XML ) it is easy to find and delete.

I think this is the only way.I will try to search other options.

Rgds

Ankit

[Former Member]

Hello..

Thanks for your replay. Since I am getting the price condition files every day which contains the materials that I don't want to import, I am not able to delete these manually. I have tried the fuction MDIS unmapped values and set it to Fail. This worked fine for a few records, bu when I imported all 4 mill records, the MDM did not seem to handle it very well. The import job stoped with a value exeption (in which is correct) but only a few records were imported.

Thanks for your help and I hope that you are able to find another sollution for this.

Regards, Anders

[Former Member]

Hello ,

If you have any doc.regarding the performance enhancement of MDM as you mentioned.Provide some more suggestions as we are also going to have such type of requirements.

Thanks

Brad

[Former Member]

Hi Anders,

We have loaded around 1.3 million main table records. My experience loading such a big quantity was not that impressive when we used Import manager. Here are the steps we followed:

1] Not possible to load all records at the same time.

2] Break the file into say 30000 - 50000 records per file.

3] Took lot time first at "Record Matching" step and then to actually impport the records.

Fortunately we were on SP04 version and we had Batch clients that time which helped me a lot. But again we dint see any performance improvement in actually importing the records. Its just we could save time in: 1. Opening Import Manager 2. Loading my saved map.

We have not tried loading such amount in latest versions. Please share your experience once done.

Regarding second question , thats not possible , you will have to change you source file for that or import the record as is with mapping "Material" unmapped values to "NULL" in the destination.

Regards,

Dheeraj.