Having trouble in restarting the SAP instance

[Former Member]

Hi Experts,

I've restarted our sap system in sapmmc and we've noticed that under j2ee process > sdm (green) > dispatcher (green) > server 0 (yellow) with the status of Starting apps. it has been more than an hour and the status is still startiing apps.

I've checked the developer trace of server 0:

.

.

.

.

.

Thu Jun 09 14:41:46 2011

Desired survivor size 32145408 bytes, new threshold 4 (max 31)

- age 1: 12451272 bytes, 12451272 total

- age 2: 12065592 bytes, 24516864 total

- age 3: 4301064 bytes, 28817928 total

- age 4: 6653504 bytes, 35471432 total

Thu Jun 09 14:46:11 2011

Desired survivor size 32145408 bytes, new threshold 4 (max 31)

- age 1: 8820536 bytes, 8820536 total

- age 2: 10923432 bytes, 19743968 total

- age 3: 11550192 bytes, 31294160 total

- age 4: 4192328 bytes, 35486488 total

Thu Jun 09 15:26:46 2011

Desired survivor size 32145408 bytes, new threshold 4 (max 31)

- age 1: 7963392 bytes, 7963392 total

- age 2: 4799528 bytes, 12762920 total

- age 3: 10711912 bytes, 23474832 total

- age 4: 11155592 bytes, 34630424 total

Why is this happening? i don't know if this will up or not? but based on our experience if the status is "Starting apps" it will surely UP. but we've waited too long and still it doesn't UP. Please help.

to give you an overview. we are using Microsoft Windows Server 2003 x64, 16GB ram.

Also, i've checked the parameters in configtool

-> instance_ID27168

> dispatcher_ID2716800

> server_ID2716850

dispatcher_ID2716800

>max heap size (MB): 170

>java parameters:

-XX:NewSize=57M

-XX:MaxNewSize=57M

-Xms170M

-XX:+DisableExplicitGC

-verbose:gc

-Xloggc:GC.log

-Djava.security.policy=.\java.policy

-Djava.security.egd=file:/dev/urandom

-Djco.jarm=1

server_ID2716850

>max heap size (MB): 2048

-Djco.jarm=1

-XX:MaxPermSize=1200M

-XX:PermSize=1200M

-Xms2048M

-Xmx2048M

-XX:NewSize=341M

-XX:MaxNewSize=341M

-XX:+DisableExplicitGC

-verbose:gc

-Xloggc:GC.log

-XX:+PrintGCDetails

-XX:+PrintGCTimeStamps

-Djava.awt.headless=true

-Dsun.io.useCanonCaches=false

-XX:SoftRefLRUPolicyMSPerMB=1

-XX:SurvivorRatio=8

-XX:TargetSurvivorRatio=90

-Djava.security.policy=./java.policy

-Djava.security.egd=file:/dev/urandom

-Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy

-Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy

-Djavax.rmi.CORBA.PortableRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy

-XX:+UseParNewGC

-XX:+PrintTenuringDistribution

-XX:+UseTLAB

-XX:+HandlePromotionFailure

-XX:+PrintClassHistogram

-XX:+HeapDumpOnOutOfMemoryError

-Xss2m

Please let me know if you have any documents or suggestions pertaining to these parameters and reason for taking too long to UP the system..

Thanks,

Tony

[Former Member · Accepted solution]

Hi,

If you have an antivirus on the server, disable on-access scan while starting up SAP Java stack.

On my systems, the startup time is halfed without on-access scan.

But yes, compared to the abap stack the Java stack is painfully slow to start up.

Some memory tuning may also help.

Regards,

Olivier

[Former Member · Accepted solution]

Thanks for that info Oliver, do you have any documentation there to help me in memory tuning ?

Regards,

Tony

[Former Member]

Those log entries you included at the beginning of your post are just output from the Java Garbage Collector.

Looks like you should have a GC.log file somewhere according to your config output.

I would go and take a look at the Server GC.log and see what it's doing.

If you know enough about Java Virtual Machines you may get some valuable info that will help you tune.

As a quick look at your Server config, you seem to have a large Permanent Generation size (-XX:MaxPermSize=1200M & -XX:PermSize=1200M), but a small New Generation size (-XX:NewSize=341M & -XX:MaxNewSize=341M).

I don't know if these are SAP recommended sizes, but they will affect the amount of memory given to objects that are newly created vs objects that have been through various tenures (garbage collections) in the Java VM.

I am imagining that the system is struggling to fit all newly created objects into the Java VM memory "slots" as the system starts, and is more than likely performing a lot of garbage collections and tenuring.

Your memory size allocations look like they would be well suited to the system once it is started and objects in the Java VM memory have matured to the Permanent Generation.

[Former Member]

Thanks paul! I will review and look into it!

Thanks,

Tony

[Former Member]

Hi Tony,

Take a look at dev_server0 and std_server0.out and see which service is taking long time to start.

Regards,

Pinkle

[Former Member]

@Soujanya - that isn't from his log file, it is from the configuration in his configtool.

@Tony - what has changed recently to cause this error? I would suggest what Soujanya says regarding optimising the memory paramaters. Also check the other log files. I have seen SAP Portals take around 1 hour to start up after major code changes have been made. Have any very large transports gone in?

[Former Member]

Hi Paul,

After restarting the server it went well. but still it took lots of time before it went UP, i think it's almost 30 mins.. anyways, thank for all your messages. We all know sometime that restarting the server is not a best practice, so i'd like to ask if you have any recommendations on how to tune up well our parameters or memory parameters in configtool? so we can prevent this long time running status in starting apps?

Thank you,

Tony

[Former Member]

Please see [PERFORMANCE BEST PRACTICES GUIDE FOR SAP NETWEAVER PORTAL 7.0|http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/f0f1358d-0812-2c10-b58c-c7bdd7a0cdce]

Note that it takes a very long time to go through and obviously you need to do it in your dev and QA environments first. I have also done a lot of testing for SAP Portal performance using LoadRunner.

Many issues in the SAP Portal are memory related and following the guide helps a lot, but it will not help reduce the startup time of the SAP Portal. We have some very fast servers, but the startup times are still extremely slow.

[Former Member]

Hi,

The log clearly says "XX:+HeapDumpOnOutOfMemoryError " . You need to do memory tuning in Config tool -> Server node. Refer note 723909.

Rgds,

Soujanya