Performance of import server is prohibitively slow

[Former Member]

Our production server imports data at a rate of 1 record/second. In development with much smaller dataset in repository, the import is 50x faster. The time is spent during the step GetMatchedNumberForRecordValues

Our table being imported has 5 key values; it has 6m rows. I know its quite large, but in DB world we have seen tables more than 6m rows....

Has anyone seen this problem? what solution did you find?

for mass or bulk import of data( MDIS OR IMPORT MANAGER ) Which one is the best ? can any tell explain clearly why ?

Thank you in advance.

Vijayakumar Maddula

[Former Member · Accepted solution]

Hi VijayaKumar.

Import Server(MDIS) is best for importing Mass or Bulk Data. Please verify this on Page 246/403 of MDM Console Reference Guide. Table 63. Conceptual Differences Between Import Manager and MDIS

http://help.sap.com/saphelp_nwmdm71/helpdata/en/4b/71608566ae3260e10000000a42189b/MDMConsole71.pdf

Maximum import file size       Limited (50,000 records) for  IM         Unlimited for MDIS

I think you need to make use of Import sleep time and Import Slice Size parameters for MDM Optimized Performance.

How to configure these Parameters for this Please refer below thread:

Also please refer, Table 59. Optional MDM Server Parameters of MDM Console Reference Guide:

parameters like: Bulk Import Silo Page 226-227/403

Hope it helps..

Regards,

Mandeep Saini

[Former Member · Accepted solution]

hi mandeep ,

Thnks for your rply , but we have already tried that option what you said ,but still no improvement . MDIS IS HANGING WHEN TRY TO IMPORT BULK DATA . There was no problem with small amount of data . when we try import like 30 mb of data ,mdis is hanging . what are the other causes for mdis to hang frequently ?

Thanks and regards ,

M.Vijaykumar.

[Former Member · Accepted solution]

Hi VijayKumar,

Other alternative i can suggest rather than using MDM Import Manager and Import Server, you can use MDM JAVA APIs to import Data. Please refer below link for more details.

http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/b0b80cb8-7f8a-2c10-b886-e7b3bccba...

Regards,

Mandeep Saini

[Former Member]

Bulk Data should be imported using Import Manager rather than Import Server. What guide is saying that is not always correct.

In real time scenarios, bulk data is always loaded using Excel format and using manual import.

Rest for performance the issue while using Import Server is resolved in MDM 7.1 SP04. Please refer SAP release note from service market place.

BR,

Alok Sharma

[Former Member]

hi Alok what you said is true . We have already applied that MDM 7.1 SP04 patch but no use . We have alreadty raised a note regarding this untill now no response ....

[Former Member]

Hi Vijay

Totally agree with Alok. For bulk import we should use excel files and import maps.

We have faced similar problems during the initial data loads/bulk load done in MDM and followed few steps to reduce the server outages. We have seen that the time taken is huge for loading the import map and matching the records-

1. reduce the file size, not more than 50k records in 1 file.

2. Optimize the import map. Fields which dont have any values need not be mapped.

3. some of the field partitioning and transformations can be done in the excel file to make the import map lighter.

4. For importing the qualified tables use separate maps and files. Delink with the main table import.

Thanks-ravi