Handling large files.

[Former Member]

Hi,

We have SAP-Business Connector 4.6;when we have a inbound xml file of size approx 7MB,the server hangs.We have to manually restart the service.Then the file is reprocessed without error after a scheduled restart.

How can i handle this without having to manually restart the service ?

Can we handle file size of 7MB in SAP-BC 4.6,if yes,where can i do the changes ?

Expert response is haighly appreciated.

Thanks in advance.

Regards.

[Former Member · Accepted solution]

Kindly have the a look at the following links....This will tell you the process to enable High volume of data transfer efficiently

/people/william.li/blog/2008/03/07/step-by-step-guide-in-processing-high-volume-messages-using-pi-71s-message-packaging

https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/9056c89c-1a2e-2a10-8a99-eae154e7...

http://help.sap.com/saphelp_nw04s/helpdata/en/42/e7bb8f7c9d3ee0e10000000a1553f6/frameset.htm

This will Help you out..................................

BR,

Sainath

[Former Member]

Thanks to Damien & Christian,problem resolved.

[Former Member]

Hi,

Use Module ..payloadzipbean to handle huze files.

Regards

[Former Member]

Hi,

You can ask your source application to send small pieces of data of 1MB each instead of sending a large file of 7MB...........

or you can ask the hardware guys to increase the RAM of your Business Connector server.

Regards,

Rajeev Gupta

[Former Member]

Thanks Rajeev for prompt reply,But the thing is the data is consolidated report of the transactions for the day.And this is sent by partner,so spliting it from source is not an option.Secondly this file is processed later after the server restart w/o any change to h/w config.

Is there any option of changing the max. file size. somewhere is what i am looking for.

Regards.

[Former Member]

How do you receive that inbound XML ? Via HTTP ?

Did you check the BC error log ? Or the server log ? Maybe there are some hints about the problem.

Probably you have to increase the RAM for the Java virtual machine. It seems that when the server starts and immediately processes that file, it has enough RAM, but after a while, when it has processed already other things, there may be not enough RAM left.

CSY

[Former Member]

Hi

As Christian suggested you can increase the memory allocated to the JVM when starting BC. This is done by changing startup script mem parameters:

<bcserver>\bin\server.bat (or server.sh in the case of unix) is the startup script

then you should see mxm mxs for max and min memory allocation.

Other things you can do are:

1. Increase the grabage collection parameter in properties.

2. Disable any large packahes that you are not using SAP BC Administrator -> Packages -> Managment.

3. Minimize logging - Settings -> Logging -> Edit Logging Settings -> Level of Logging Field -> change this to '1'

I am telling you from memory as I have faced similar issues on SAP BC so cannot give specifics as I do not have a BC access at the moment.

Thanks

Damien

[Former Member]

Thanks Damien,

Would also like to thank all the above for their expert advise.

While going throught the server.bat(Windows) file i have seen some settings like this in my file:

set USEJVIEW=false

rem set JAVA_MIN_MEM=890M

rem set JAVA_MAX_MEM=990M

rem set JAVA_MIN_MEM=1800M

rem set JAVA_MAX_MEM=1900M

rem set JAVA_MIN_MEM=1000M

rem set JAVA_MAX_MEM=1600M

set JAVA_MIN_MEM=1000M

set JAVA_MAX_MEM=1400M

I cannot find anything as mxm or mxs;are you pointing to change these parameters?..which one?My file size is 7MB ,which should be handled as per the above setting.Isn't it?

Awaiting your pointer on this.

[Former Member]

He refers to

set JAVA_MIN_MEM=1000M

set JAVA_MAX_MEM=1400M

>file size is 7MB ,which should be handled as per the above setting.Isn't it?

that depends on the load of the system, other parallel requests. Also, some XML parsers need huge amounts of RAM. Just try to open a 7MB XML file in Internet Explorer. And check the needed RAM in the task manager. You will be surprised.

How much total physical memory do you have on the Business Connector machine ? Is the BC the only server application ? How much free RAM shows your system when the BC is stopped ?

CSY

[Former Member]

HI Vipul

Yes these are the settings (sorry as I said I did not have access to SAPBC and working from memory ). Looks like some one has already updated the values because the defaults are 256 and 512 and as can be see there are numerous lines that have been removed using "rem". The current settings:

set JAVA_MIN_MEM=1000M

set JAVA_MAX_MEM=1400M

It is likely you can increase these further. Check ont the machine the mem available and update accordingly. If this machine is used only for SAP BC then you can increase to the max. Also in the past I have set the MIN and MAX to the same "large" value. I would point out theough on SAP BC for very large files e,g, 30M you may never overcome the performance issues. It might be a better option to split the file in a pre processing step if this is possible. Also It might be worth looking into SAP BC 4.8. This is the latest version and performance may have inproved. Now that I am thinking about there is another thing that you can do. That is to ensure that before processing the large file that you delete all the "old" transactions n BC. This can be done manually through BC Administrator and also there are automated jobs that can be scheduled to delte these old transactions. This does help performance considerably.

Thanks

Damien

[Former Member]

Hi Chritian & Damien,

Both of you pointed out the same thing, as per your inputs:

>How much total physical memory do you have on the Business Connector machine ?

6GB approx.

>Is the BC the only server application ?

Yes.

How much free RAM shows your system when the BC is stopped ?

BC Stopped- 0.2G BC Started-1.3G Total- 6.6G

I am planning to do the suggested changes,and will let you know the update.

We are already in talks with our seniour-management for upgrading it to latest 4.8 version.

Thanks for your suggestions.

Regards.

Vipul.