How to process huge files using File Adapter without ESR and no FCC?

[santosh_k3]

Hi All,

I came to a scenario where i need to pick the file from R/3 sys which of around 500MB and need to transport it to the Third party sys using PI 7.1 . This is just simply a pass through data i.e.., no mapping is required (no ESR). and i don't to make use of FCC or BPM

My question is can the File Adapter can pick the file which is of 500MB size?

If not then what would be best solution to do it?

Do i need to divide the file into smaller sizes say around 100MB a file each in R/3 sys and then pick those files .

can i do without dividing the files in the R/3 sys?

Thanks in advance.

Sai

[Former Member · Accepted solution]

Hi Sai,

Technically, PI can take any amount on the file, but it all depends on the hardware you are having on the system. So you should not have a problem with 500MB. Else you can see for other options like zipping the file and sending. Zip the file and see how big it is.

Regards,

---Satish

[baskar_gopalakrishnan2 · Accepted solution]

Even with great hardware support still IMO, processing 500 MBfile is a big effort. Also do you think zipping the file would reduce the load?

[Former Member · Accepted solution]

Hi Bhaskar,

If you zip probably the size would come less which may increase his performance a little. But I feel SAP also dont recommend this much load with PI.

If I am in his shoes then I will do a simple FTP without using PI itself. Else will look into other options. Thats my two cents.

Hope this alexs blog will give an idea to Sai:

/people/alessandro.guarneri/blog/2007/02/21/sap-xi-acting-as-a-huge-file-mover

Regards,

---Satish

Edited by: Satish Reddy on Jul 20, 2011 3:02 PM

[former_member854360]

Hi ,

If it is a only file transfer then use any FTP tool ........because transfering 500mb file through PI will be performance intensive.

[Former Member]

Hi Sai,

You can check options of BATCH SCRIPT, that needs to be called to move your 500 MB size file from ECC to third party. To get the batch scripts to work for 3rd party you need to check with your network team. You can schedue a btach job on ECC to run this script periodically. OR configure a sender file channel with option RUN OS Command before message processing, the channel polls for a file which does not exist on directory. Every poll of the channel triggers OS command and process if any files are found by batch script.

Cheers

Veera

[santosh_k3]

Hi All,

Thanks to everyone for your valuable suggestions,

The problem is solved by transferring the file without using PI .

ie.., directly the file is transferred to FTP

Thanks

Sai

[rajasekhar_reddy14]

Hi,

at one shot if you want process 500 MB then dont use PI try other options.

Alex solution feasible in your case, even if Zip the file i dont think the size will come down.

Regards,

Raj

[Former Member]

Hi Sai,

Why can't you use PayloadZipBean in the module processor, that compresses the payload.

Ref: http://help.sap.com/saphelp_nwpi711/helpdata/en/45/da9358a1772e97e10000000a155369/content.htm

Thanks,

[agasthuri_doss]

Sai,

>>My question is can the File Adapter can pick the file which is of 500MB size?

As one message it is not good idea, As you mentioned can break/chunk and send it. Have each broken message with tag (file name or in header ) so that you can figure out the sequence order. Yes you can do in ID alone with out ESR.

Cheers

Agasthuri

[santosh_k3]

Hi Agasthuri,

Thanks for u r reply.

one more question was can i do it without dividing the file in R/3 side.

what would be the best solution in this case.

thanks & regards

sai.

[agasthuri_doss]

Sai,

My Understanding as follows, Since No Mapping that is no transformation & you want to post it in SAP ECC system right, If that so try LSMW from SAP ECC side, It can handle more than 500 mb file too

Cheers

Agasthuri

[santosh_k3]

Hi Agasthuri,

No, my scenario is to pick the file(500 MB) from ECC and place the file in third-party sys.

(ECC)File->PI--->File(third party).

thanks&regards

sai

[baskar_gopalakrishnan2]

500 MB is a huge file to process. Though you dont involve mapping and no esr involved, still it might cause issue in performance.

Run it in the off peak schedule. TO use design interface without ESR , refer this [link|http://www.sdn.sap.com/irj/scn/weblogs?blog=/pub/wlg/4383] [original link is broken] [original link is broken] [original link is broken];.

[santosh_k3]

Hi Baskar,

thanks for u r reply.

Run it in the off peak schedule.

I did not get u , could u please describe on this.

Thanks

sai

[Former Member]

processing 500 mb of data in PI is bit dicey...why dont u try out Agasthuri's suggestion??

[naveen_chichili]

Hi Sai,

Processing 500MB file causes performance issues ..........my best suggestion is to process the data in chunks ...

If you process at a single shot it effects the other interfaces because of performance issues.....though you increase the tuning parameters in PI server...this is not a best practice...

Regards,

Naveen.