Portal restarted automatically

[Former Member]

Hi ,

My portal nodes are getting restarted frequently and please see the entries in dev_dispatcher during the time.

af type="nursery" id="13" timestamp="Jul 11 15:11:07 2011" intervalms="66837.856">

<minimum requested_bytes="18760" />

<time exclusiveaccessms="0.155" />

<nursery freebytes="0" totalbytes="53373952" percent="0" />

<tenured freebytes="221537256" totalbytes="241172480" percent="91" >

<soa freebytes="209478632" totalbytes="229113856" percent="91" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<gc type="scavenger" id="13" totalid="13" intervalms="66839.002">

<flipped objectcount="59091" bytes="9878080" />

<tenured objectcount="9226" bytes="627872" />

<refs_cleared soft="0" weak="0" phantom="0" />

<finalization objectsqueued="473" />

<scavenger tiltratio="74" />

<nursery freebytes="43573392" totalbytes="55013376" percent="79" tenureage="7" />

<tenured freebytes="220095464" totalbytes="241172480" percent="91" >

<soa freebytes="208036840" totalbytes="229113856" percent="90" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<time totalms="16.468" />

</gc>

<nursery freebytes="43554632" totalbytes="55013376" percent="79" />

<tenured freebytes="220095464" totalbytes="241172480" percent="91" >

<soa freebytes="208036840" totalbytes="229113856" percent="90" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<time totalms="17.942" />

</af>

Mon Jul 11 15:11:58 2011

<af type="nursery" id="14" timestamp="Jul 11 15:11:58 2011" intervalms="50358.301">

<minimum requested_bytes="4120" />

<time exclusiveaccessms="0.156" />

<nursery freebytes="0" totalbytes="55013376" percent="0" />

<tenured freebytes="220095464" totalbytes="241172480" percent="91" >

<soa freebytes="208036840" totalbytes="229113856" percent="90" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<gc type="scavenger" id="14" totalid="14" intervalms="50359.689">

<flipped objectcount="37010" bytes="7459008" />

<tenured objectcount="20230" bytes="1585344" />

<refs_cleared soft="0" weak="0" phantom="0" />

<finalization objectsqueued="400" />

<scavenger tiltratio="76" />

<nursery freebytes="47886392" totalbytes="56398848" percent="84" tenureage="7" />

<tenured freebytes="217867216" totalbytes="241172480" percent="90" >

<soa freebytes="205808592" totalbytes="229113856" percent="89" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<time totalms="25.459" />

</gc>

<nursery freebytes="47882272" totalbytes="56398848" percent="84" />

<tenured freebytes="217867216" totalbytes="241172480" percent="90" >

<soa freebytes="205808592" totalbytes="229113856" percent="89" />

<loa freebytes="12058624" totalbytes="12058624" percent="100" />

</tenured>

<time totalms="26.244" />

</af>

Also fine the entries in default trace

#

#1.#00215E3437320246000000020001316C0004A7CB9501D5E9#1310392726967#com.sap.ip.collaboration.rtc#sap.com/irj#com.sap.ip.collaboration.rtc.class com.sap.ip.col

laboration.core.api.rtmf.core.RTMFMessaging.JMSPolling.run()#Guest#0##n/a##e5936d51abc511e0979500215e343732#Thread[Thread-650,5,SAPEngine_Application_Thread[i

mpl:3]_Group]##0#0#Error##Java###!!!!!!!!!!!!!!!!! restarting JMS !!!!!!!!!!!!!!!!!!!!!!!!!:

[EXCEPTION]

#1#javax.jms.JMSException: Cannot assign connection context.

at com.sap.jms.protocol.notification.ServerExceptionResponse.getException(ServerExceptionResponse.java:271)

at com.sap.jms.client.session.Session.checkReceivedPacket(Session.java:2656)

at com.sap.jms.client.session.Session.createProducer(Session.java:2437)

at com.sap.jms.client.session.QueueSession.createSender(QueueSession.java:81)

at com.sap.ip.collaboration.core.api.rtmf.core.RTMFMessaging$JMSPolling.run(RTMFMessaging.java:1189)

at java.lang.Thread.run(Thread.java:770)

Any hints what i can look for so its not repeated ?

Thanks

Edited by: Anish John on Jul 11, 2011 8:09 PM

[Former Member]

HI,

Check on memory parameter in the Server.

Using Cofig tool and Visual Admin Increase the size for virtual machne in server.

Increase the memory parameters

if this is help full vote a credit point.

Thanks

Santhose

[Former Member]

Hi,

I got something from dev_jcontrol as well

JStartupIReadSection: read node properties [ID213241451]

-> node name : server1

-> node type : server

-> node execute : yes

-> jlaunch parameters :

-> java path : /usr/java14_64

-> java parameters : -Xmn1000M -Xgcpolicy:gencon -verbose:gc -Xj9 -Xdisableexplicitgc -Djava.net.preferIPv4Stack=true -Xdump:system:events=systhrow,filter=

java/lang/OutOfMemoryError,request=exclusive+prepwalk -Xsoftrefthreshold1 -Djco.jarm=1 -Djava.security.policy=./java.policy -Djava.security.egd=file:/dev/uran

dom -Dorg.omg.CORBA.ORBClass=com.sap.engine.system.ORBProxy -Dorg.omg.CORBA.ORBSingletonClass=com.sap.engine.system.ORBSingletonProxy -Djavax.rmi.CORBA.Portab

leRemoteObjectClass=com.sap.engine.system.PortableRemoteObjectProxy -Dorg.omg.PortableInterceptor.ORBInitializerClass.com.sap.engine.services.ts.jts.ots.Porta

bleInterceptor.JTSInitializer -Djava.awt.headless=true -Xbootclasspath/p:/usr/sap/DDA/SMDA97/SMDAgent/applications.config/com.sap.smd.agent.application.wily/B

ytecodeAgent/ISAGENT.8.1.0.0-2009-09-04/wily/connectors/AutoProbeConnector.jar:/usr/sap/DDA/SMDA97/SMDAgent/applications.config/com.sap.smd.agent.application.

wily/BytecodeAgent/ISAGENT.8.1.0.0-2009-09-04/wily/Agent.jar -Dcom.wily.autoprobe.prependToJVMRuntimePath=/usr/java14_64/jre/lib/jclSC14/classes.zip -Dcom.wil

[Former Member]

have you checked std_server<server_node>.out (example: std_server0.out) log file from instance work directory. How many server nodes you have per instance and is the memory is sufficient for the load?

I can see you have Wily configured, can you please analyze the memory utilization from Wily if possible.

[Former Member]

Hi sunil,

Thanks for your reply . we have 3 server nodes in that instance and you are write- wily introscope is configured as I put in the log before .

I am getting the same kind of errors in std_server*.out as well . Also I strongly believe it should be related to heap memory !! But I need to prove it and thats what I am looking for

Tks

Edited by: Anish John on Jul 11, 2011 9:28 PM

[Former Member]

So far I can't see any memory problem, the dispatcher log has some normal garbage collection and the jcontrol log just shows some standard parameters, one of them is '-Xdump:system:events=systhrow,filter=java/lang/OutOfMemoryError'.

The default trace shows part of the problem, but it would be interesting to see what happened before and corresponding parts of the server node where the error comes up.