cancel
Showing results for 
Search instead for 
Did you mean: 

CSV to XML for Huge File

Former Member
0 Kudos

Hello Experts,

I have a scenario where in I have to use a sender SFTP adapter and convert 25 MB CSV file to XML and map it to IDOC.

I have used message transform Bean to do the conversion,but the load on the PI system too much.Is there an alternate approach for this

1 more approach is that ,I can read the entire row into one field and then use java mapping to extract the fields based on comma separator.

Even then the load is too high.

Please suggest any alternate approach

Regards

Advit Ramesh

Accepted Solutions (1)

Accepted Solutions (1)

engswee
Active Contributor
0 Kudos

Edit: Praveen got to this a minute before me! I should probably stop answering threads as I'm never fast enough

Hi Advit

If your CSV file can be broken down into separate recordset, then you can use the RecordSet Per Message feature in FCC so split the file into separate messages.

Patch 02 of SP04 of the SFTP adapter contains built in FCC functionality that supports the feature. This was you don't need to use MTB which do not have this splitting feature. Refer SAP Note 2144272 for the details of the patch - make sure you also update the Adapter Metadata in ESR.

Processing such a big CSV file into XML in one single message will always be a challenge as the size of the converted XML is too big and cause slowdown due to garbage processing in the JVM.

Regards

Eng Swee

Message was edited by: Eng Swee Yeoh

Answers (1)

Answers (1)

former_member182412
Active Contributor
0 Kudos

Hi Advit,

Use Record Sets Per Message and then adapter split the file into multiple messages then you can create different idocs for individual messages.

The below blog is example for file adapter but you can do the normal file content conversion for latest SFTP adapter because latest SFTP adapter support file content conversion like file adapter. No need to use message transform bean for FCC.

Night Mare-Processing huge files in SAP XI | SCN

Regards,

Praveen.