Master data import from FTP

[Former Member]

Hi All,

We are implementing Sourcing 9 SP15. We have exported master data related to Vendor in FTP. We have configured schedule task in Sourcing to pick the file from FTP and import in S2C.

Scenario1:- when we are trying to import file from FTP which has 1200 records. Able to pick from FTP and imported successfully. I can see import finish status summary inside system administration->import.

Svenario2:- this time we are trying to import same master data but we have increased the record number to 13000 thousand. In this scenario we are facing issue. Import task triggered and it's keep running. The worst part is system automatically started new import task for the same file after sometime. Please find below screenshot for your reference.

Due to the above issue system started giving memory dump. So we deleted the file from the queue folder inside application server folder structure and filed from the temp folder inside <SID> ladder but no luck.

Please let me know if anybody has faced this issue in past or had any idea on how we can handle this issue.

Cheers,

Deepak

[Former Member]

Hi Deepak,

Can you please try importing records in 1500 entries each in 8 steps and let me know what happens this time.

Regards,

Ravi

[Former Member]

Hi Ravi,

Thanks for reply.

I have uploaded 8000+ Vendor record in one go in past. I tired importing 4000 record and FTP import task is going in loop. System is adding new import task in every 20 minutes for the same master data file which is killing memory of application server.

Here issue is, how import schedule task is going in loop and add new import task for the same vendor file from queue folder during FTP import.

Cheers!!!

Deepak

[Yateesh_h]

Hi Deepak,

Could it be because of the Period Length setting in Schedule Task setting set to 20 minutes?

Please check.

Cheers,

Yateesh

[Former Member]

Hi Yatessh,

Thanks for reply.

We have defined schedule task Frequency as Run Once so from configuration point of you i think no error.

For testing purpose we added wrong vendor IDs in the Vendor-external system mapping CSV file to check if import program is picking the record for import or not. Please find below observation:-

• In the background events  inside system administration i can see system is able to read last record of the file
• Tested import impact on the existing supplier master and replicated flag became true for the correct supplier IDs
• Import program again started and same error started recording for in-correct supplier in background error report.
  

Cheers,

Deepak

[Former Member]

Hello Deepak,

     What are the memory settings for the JVM. Did you check the log files to see if there are any errors being reported on the vendor records being imported. Can you also check the log files it may give you an idea why the scheduled task is restarting (This usually happens if the running task teerminates for some reason unable to do it job..). you should not delete any files when it its doing this as it will only make the situation worse.. the scheduled task will re-try for a few times and fail eventually.

   There is a blog on how the scheduled tasks work. That may give you some detials about the inner workings.

    For now you should look in the logs and figure out why the task is failing and take care of it.

Regards

Prasad

[Former Member]

Hi Prasad,

Thanks for reply.

We reported this issue to SAP and they are also investigating. Please find more detail on the issue:-

• First time when we tried to import 11000 vendor record due to memory issue application server went down. We restarted the server and import task went on loop. We waited for 1 day but no luck. Then we reported issue to SAP and they have provided steps on how we can remove file from the "queue" folder & binary files from the temp folders.
• Then we tried to import vendor master with 60, 157 & 1200 records & it worked fine.
• Later we tried with 4000 record and again import stuck in loop.

My basis team has increased the memory of the server by 30GB but no luck.

One point to share, if we write script which will validate the supplier master when it is going to save may cause any harm to FTP import because we wrote a script on Vendor class which validate uniqueness of one extension. I have seen background task where system has reported issue based on the exception we wrote in script as expected.

When i tried to import record with error to check my script it worked the way we want. It has reported error in background error and import rest of the correct record. I tested with 1200 record where 150 were having duplicate value for the extension field. System took 3 minutes to process 1200 record then why going in loop for 4000+ records

Cheers,

Deepak

[ginodw2014]

Hi Deepak,

This is an system issue, we also had the same issue after upgrading from v7 to v9 sp16 (import user's from ERP to SAP Sourcing). SAP also investigated and basically responded that we reduce the file to have lesser records and do imports in bunches. So hopefully they will include a fix for this in the next sp release.

Regards

Gino