JsfOpenShm failed error .

[Former Member]

Hi Jim Farney ,

We have intially installed the single instance (J00) with the SCS instance on a single hostname in the SAP NW CE 7.2

Then we have installed another instance (J02) .

(i.e )

Qc1 --> Host

SCS01 -- SCS

J00 -- Application Server

J02 -- Application Server (additional )

After installing the another additional application server , after making a restart , when i try to acess the url : http ://host :portnumber .

Suddenly it cannot able to reach the port number : 50000

When i checked in jsmon , its not allowing me to check the process .

Its giving the below error :

JsfOpenShm failed: object not found

Cannot access shared memory.

Kindly suggest for the solution .

Thanks

S SURESH

[former_member371327]

Hello,

is there some fix for the issue?

[markus_doehr2]

Are both instances started and running?

Did you configure the memory parameters so that both instances are able to run simultaneously?

Markus

[Former Member]

Hi Markus ,

Thanks for the reply .

i cannot able to check in jsmon about the instance status .

Both instance were started & changed into the status stopping ( this i can able to server0 logs )

Memory Parameter has been added to both instance because we made replica .

Intial Heap Size = 2048

Max Heap Size = 2048

Perm Size = 1024

Max Perm Size = 1024

Both the instance are having the same value of memory parameter .

Thanks

S SURESH

[Former Member]

Hi Markus ,

For the additional information , i have attached dev_jcontrol log file .

Thr 47483395225696 *** ERROR => SfCreateShm: Shm in use by process 7106. http://jsfxxshm_mt. 1219

Thr 47483395225696 *** ERROR => Can't create static shared memory (rc = 13 already in use) http://jcntrxx_mt.c 2001

Thr 47483395225696 *** ERROR => Can't initialize JControl Administration http://jcntrxx_mt.c 293

Thr 47483395225696 JControlCloseProgram: started (exitcode = -1)

Thr 47483395225696 *** ERROR => SfDestroyDynShm: Dynamic shared memory is not opened. http://jsfxxshm_mt. 2298

Thr 47483395225696 *** ERROR => SfDestroyShm: Static shared memory is not opened. http://jsfxxshm_mt. 1326

Thr 47483395225696 JControlCloseProgram: good bye... (exitcode = -1)

Thanks

S SURESH

[Former Member]

Hi Suresh,

Please do the following:

1) Stop the SAP system.

2) Do a cleanipc <sysnr> remove, where sysnr will be replaced by 00,01 and 02. So three executions.

3) Type "ipcs -a". See if there are any shared memory segments visible. Especially ones with the <sid>adm users.

4) If you see that there are shared memory segments present, please reboot your host once.

5) Start SAP system again.

Please let me know if this works out.

Thanks,

Deoraj.

[Former Member]

Hi Deoraj ,

Thanks for the reply .

I have already tried this method . this does not help .

Thanks

S SURESH

[markus_doehr2]

It seems that your reaching operating system imposed limits.

Is there enough physical memory for those instances available?

Markus

[Former Member]

Hi Markus ,

Thanks for the reply

We have enough total physical memory of 20 gb & Swap Space of 40 gb .

As a try i have uninstalled the additional application server (J02 ) , then the system is working fine , but after installing ( J02 ) additional instance i am gettting the error & even the url is not working .

Thanks

S SURESH

[markus_doehr2]

> As a try i have uninstalled the additional application server (J02 ) , then the system is working fine , but after installing ( J02 ) additional instance i am gettting the error & even the url is not working .

jcontrol is not allowed to allocate an additional share memory segment.

I suggest you check the OS kernel parameter for an additional system using kctune. They may need to be increased.

Markus

[Former Member]

Hi Markus ,

Thanks for the reply .

We have given shmmax to 22 gb before itself .

Kindly suggest which OS kernel parameter has to be changed .

Thanks

S SURESH

[Former Member]

Hi Markus ,

Any update on the message for which kernel paramter has to be changed in the OS level

Thanks

S SURESH

[Former Member]

Hi Markus ,

I am attaching dev_icm logs

-

---

trc file: "dev_icm", trc level: 1, release: "720"

-

---

sysno 00

sid SCE

systemid 390 (AMD/Intel x86_64 with Linux)

relno 7200

patchlevel 0

patchno 45

intno 20020600

make multithreaded, Unicode, 64 bit, optimized

profile /usr/sap/SCE/SYS/profile/SCE_J00_ustrlxqa08

pid 23339

[Thr 139664033961808] Tue Jun 14 07:25:55 2011

[Thr 139664033961808] Parameter system/type has value: "J2EE"

[Thr 139664033961808] *** WARNING => The maximum number of sockets supported on this host is 1018.

This is less than the number of sockets configured in parameter icm/max_sockets (2048) [icxxman_mt.c 3305]

[Thr 139664033961808] started security log to file ./dev_icm_sec

[Thr 139664033961808] SigISetDefaultAction : default handling for signal SIGCHLD

[Thr 139664033961808] ICM running on: ustrlxqa08.genre.com

[Thr 139664033961808] Profile configuration error detected, use temporary corrected setup

[Thr 139664033961808] Shared Pool 10: ipc/shm_psize_10 = 79691776 (too small)

[Thr 139664033961808] Shared Pool 10: (smaller than min requirement 132565864)

[Thr 139664033961808] Shared Pool 10: (estimated size assumed 136000000)

[Thr 139664033961808] MtxInit: 30001 0 2

[Thr 139664033961808] ***LOG IM1=> IcmInit, Startup (ICM&ustrlxqa08.genre.com&23339&) [icxxman_mt.c 1873]

[Thr 139664033961808] IcmInit: listening to admin port: 65000

[Thr 139664033961808] MPI: dynamic quotas disabled.

[Thr 139664033961808] MPI init: pipes=4000 buffers=1279 reserved=383 quota=10%

[Thr 139664033961808] CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.

[Thr 139664033961808] CCMS: Initalized shared memory of size 60000000 for monitoring segment.

[Thr 139664033961808] CCMS: Checking Downtime Configuration of Monitoring Segment.

[Thr 139664033961808] IcmInit: icm running without ABAP stack

[Thr 139663710505296] IcmProxyWatchDog: proxy watchdog started

[Thr 139663698016592] IcmJSessCommThread started

[Thr 139663697131856] IcmMplxThread started

[Thr 139664033961808] IcmJsfAttach: attached to Java Startup Framework

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 0

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 1

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 2

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 3

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 4

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 5

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 6

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 7

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 8

[Thr 139664033961808] IcmCreateWorkerThreads: created worker thread 9

[Thr 139663665576272] IcmMpiWatchDogThread: MPI watchdog started

[Thr 139663657183568] IcmWatchDogThread: watchdog started

[Thr 139663697131856] JNCMIHttpMsPutLogon: http port information not available

[Thr 139663697131856] JNCMIP4MsPutLogon: p4 port information not available

[Thr 139663697131856] JNCMIIIOPMsPutLogon: iiop port information not available

[Thr 139663648790864] ISC: created 400 MB disk cache.

[Thr 139663648790864] ISC: created 50 MB memory cache.

[Thr 139663648790864] HttpISubHandlerAdd: Added handler HttpCacheHandler(7f05f40156e0), slot=0, flags=12293) for /, active: 1, table 7f05f4015240

[Thr 139663648790864] HttpExtractArchive: files from archive /usr/sap/SCE/J00/exe/icmadmin.SAR in directory /usr/sap/SCE/J00/data/icmandir are up to date

[Thr 139663648790864] HttpISubHandlerAdd: Added handler HttpAdminHandler(10e20a0), slot=1, flags=36869) for /sap/admin, active: 1, table 7f05f4015240

[Thr 139663648790864] HttpISubHandlerAdd: Added handler HttpModHandler(10e21a0), slot=2, flags=12293) for /, active: 1, table 7f05f4015240

[Thr 139663648790864] CsiInit(): Initializing the Content Scan Interface

[Thr 139663648790864] AMD/Intel x86_64 with Linux (mt,unicode,SAP_CHAR/size_t/void* = 16/64/64)

[Thr 139663648790864] CsiInit(): CSA_LIB = "/usr/sap/SCE/J00/exe/libsapcsa.so"

[Thr 139663648790864] HttpISubHandlerAdd: Added handler HttpAuthHandler(10e2700), slot=3, flags=12293) for /, active: 1, table 7f05f4015240

[Thr 139663648790864] ICM running without ABAP

[Thr 139663648790864] HttpISubHandlerAdd: Added handler HttpJ2EE2Handler(11067c0), slot=4, flags=1060869) for /, active: 1, table 7f05f4015240

[Thr 139663648790864] Started service 50000 for protocol HTTP on host "ustrlxqa08.genre.com"(on all adapters) (processing timeout=600, keep_alive_timeout=60)

[Thr 139663648790864] Started service 50004 for protocol P4 on host "ustrlxqa08.genre.com"(on all adapters) (processing timeout=60, keep_alive_timeout=60)

[Thr 139663648790864] Started service 50007 for protocol IIOP on host "ustrlxqa08.genre.com"(on all adapters) (processing timeout=60, keep_alive_timeout=60)

[Thr 139663648790864] Started service 50008 for protocol TELNET on host "localhost"(only on this adapter) (processing timeout=60, keep_alive_timeout=60)

[Thr 139663697131856] JNCMIHttpMsPutLogon: delete http logon information

[Thr 139663697131856] JNCMIP4MsPutLogon: reset p4 logon information

[Thr 139663697131856] JNCMIIIOPMsPutLogon: reset iiop logon information

[Thr 139663697131856] JNCMITelnetMsPutLogon: reset telnet logon information

[Thr 139663648790864] IcmNetCheck: network check passed without detecting problems

[Thr 139663697131856] Tue Jun 14 07:25:56 2011

[Thr 139663697131856] JNCMIHttpMsPutLogon: delete http logon information

[Thr 139663697131856] JNCMIP4MsPutLogon: reset p4 logon information

[Thr 139663697131856] JNCMIIIOPMsPutLogon: reset iiop logon information

[Thr 139663697131856] JNCMITelnetMsPutLogon: reset telnet logon information

[Thr 139663697131856] JNCMIHttpMsPutLogon: delete http logon information

[Thr 139663697131856] JNCMIP4MsPutLogon: reset p4 logon information

[Thr 139663697131856] JNCMIIIOPMsPutLogon: reset iiop logon information

[Thr 139663697131856] JNCMITelnetMsPutLogon: reset telnet logon information

[Thr 139663697131856] Tue Jun 14 07:27:09 2011

[Thr 139663697131856] JNCMIP4CallLBListener: node 4658150, service P4 joins load balancing

[Thr 139663697131856] JNCMIP4MsPutLogon: set p4 logon port (port:50004) (lbcount: 1)

[Thr 139663697131856] IcmJSessCommServiceJoin: server 4658150 started protocol P4 (active services: 1)

[Thr 139663697131856] JNCMIIIOPCallLBListener: node 4658150, service IIOP joins load balancing

[Thr 139663697131856] JNCMIIIOPMsPutLogon: set iiop logon port (port:50007) (lbcount: 1)

[Thr 139663697131856] IcmJSessCommServiceJoin: server 4658150 started protocol IIOP (active services: 3)

[Thr 139663697131856] JNCMITelnetCallLBListener: node 4658150, service Telnet joins load balancing

[Thr 139663697131856] JNCMITelnetMsPutLogon: set telnet logon port (port:50008) (lbcount: 1)

[Thr 139663697131856] IcmJSessCommServiceJoin: server 4658150 started protocol TELNET (active services: 7)

[Thr 139663698016592] IcmJSessCommAttach: Attached to J2EE request queue SCE_00_4658150_P4

[Thr 139663697131856] Tue Jun 14 07:29:04 2011

[Thr 139663697131856] JNCMIHttpCallLBListener: node 4658150, service Http joins load balancing

[Thr 139663697131856] HttpJ2EELbPut: server 4658150 started protocol HTTP, attached to request queue SCE_00_4658150_HTTP

[Thr 139663697131856] JNCMIHttpMsPutLogon: set http logon port (port:50000) (lbcount: 1)

[Thr 139663668746576] Tue Jun 14 07:42:00 2011

[Thr 139663668746576] *** ERROR => Error in HTTP Request: 9 [http_plg_mt.c 5534]

[Thr 139663668746576] CONNECTION (id=2/5909):

used: 1, type: default, role: Server(1), stateful: 0

NI_HDL: 131, protocol: HTTP(1)

local host: 10.51.91.3:50000 ()

remote host: 10.51.91.3:45150 ()

status: READ_REQUEST

connect time: 14.06.2011 07:42:00

MPI request: <12a> MPI response: <12b>

request_buf_size: 65464 response_buf_size: 0

request_buf_used: 5 response_buf_used: 0

request_buf_offset: 0 response_buf_offset: 0

[Thr 139663668746576] Address Offset REQUEST:

[Thr 139663668746576] -

---

[Thr 139663668746576] 7f0601c88838 000000 300d0a0d 0a |0.... |

[Thr 139663668746576] -

---

[Thr 139663668746576] *** ERROR => HttpPlugInHandleNetData: HttpParseRequestHeader failed (rc=701) [http_plg_mt. 2235]

[Thr 139663667161424] Tue Jun 14 08:00:02 2011

[Thr 139663667161424] *** ERROR => Error in HTTP Request: 9 [http_plg_mt.c 5534]

[Thr 139663667161424] CONNECTION (id=2/17672):

used: 1, type: default, role: Server(1), stateful: 0

NI_HDL: 134, protocol: HTTP(1)

local host: 10.51.91.3:50000 ()

remote host: 10.51.91.3:35177 ()

status: READ_REQUEST

connect time: 14.06.2011 08:00:02

MPI request: <16e> MPI response: <16f>

request_buf_size: 65464 response_buf_size: 0

request_buf_used: 5 response_buf_used: 0

request_buf_offset: 0 response_buf_offset: 0

[Thr 139663667161424] Address Offset REQUEST:

[Thr 139663667161424] -

---

[Thr 139663667161424] 7f0601af82c0 000000 300d0a0d 0a |0.... |

[Thr 139663667161424] -

---

[Thr 139663667161424] *** ERROR => HttpPlugInHandleNetData: HttpParseRequestHeader failed (rc=701) [http_plg_mt. 2235]

[Thr 139663697131856] Tue Jun 14 08:33:31 2011

[Thr 139663697131856] JNCMIHttpMsPutLogon: set http logon port (port:50000) (lbcount: 0)

[Thr 139663697131856] JNCMIP4MsPutLogon: set p4 logon port (port:50004) (lbcount: 0)

[Thr 139663697131856] JNCMIIIOPMsPutLogon: set iiop logon port (port:50007) (lbcount: 0)

[Thr 139663697131856] JNCMITelnetMsPutLogon: set telnet logon port (port:50008) (lbcount: 0)

[Thr 139663697131856] JNCMIHttpCallLBListener: node 4658150, service Http is removed from load balancing

[Thr 139663697131856] HttpJ2EELbStopLb: server 4658150, protocol HTTP removed from load balancing.

[Thr 139663697131856] JNCMIP4CallLBListener: node 4658150, service P4 is removed from load balancing

[Thr 139663697131856] IcmJSessCommServiceStopLb: server 4658150, stop load balancing protocol P4 (active services: 7, lb restricted services: 1)

[Thr 139663697131856] JNCMIIIOPCallLBListener: node 4658150, service IIOP is removed from load balancing

[Thr 139663697131856] IcmJSessCommServiceStopLb: server 4658150, stop load balancing protocol IIOP (active services: 7, lb restricted services: 3)

[Thr 139663697131856] JNCMITelnetCallLBListener: node 4658150, service Telnet is removed from load balancing

[Thr 139663697131856] IcmJSessCommServiceStopLb: server 4658150, stop load balancing protocol TELNET (active services: 7, lb restricted services: 7)

[Thr 139663657183568] Tue Jun 14 08:33:35 2011

[Thr 139663657183568] IcmWatchDogThread: Soft Shutdown request received

[Thr 139663697131856] Tue Jun 14 08:33:46 2011

[Thr 139663697131856] JNCMITelnetCallLBListener: node 4658150, service Telnet is stopped

[Thr 139663697131856] JNCMITelnetMsPutLogon: reset telnet logon information

[Thr 139663697131856] IcmJSessCommServiceLost: server 4658150 lost protocol TELNET(active services: 3, lb restricted services 3)

[Thr 139663697131856] JNCMIIIOPCallLBListener: node 4658150, service IIOP is stopped

[Thr 139663697131856] IIOPRemoveConnsOfServer: remove all open conns for server: 4658150

[Thr 139663697131856] JNCMIIIOPMsPutLogon: reset iiop logon information

[Thr 139663697131856] IcmJSessCommServiceLost: server 4658150 lost protocol IIOP(active services: 1, lb restricted services 1)

[Thr 139663697131856] Tue Jun 14 08:33:57 2011

[Thr 139663697131856] JNCMIP4CallLBListener: node 4658150, service P4 is stopped

[Thr 139663697131856] JNCMIP4MsPutLogon: reset p4 logon information

[Thr 139663697131856] IcmJSessCommServiceLost: server 4658150 lost protocol P4(active services: 0, lb restricted services 0)

[Thr 139663697131856] JNCMIHttpCallLBListener: node 4658150, service Http is stopped

[Thr 139663697131856] HttpJ2EELbDel: server 4658150 lost protocol HTTP

[Thr 139663697131856] JNCMIHttpMsPutLogon: delete http logon information

[Thr 139663697131856] Tue Jun 14 08:34:02 2011

[Thr 139663697131856] JNCMIHttpMsPutLogon: delete http logon information

[Thr 139663697131856] JNCMIP4MsPutLogon: reset p4 logon information

[Thr 139663697131856] JNCMIIIOPMsPutLogon: reset iiop logon information

[Thr 139663697131856] JNCMITelnetMsPutLogon: reset telnet logon information

[Thr 139664033961808] Deactivated service 50000 for protocol HTTP on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139663697131856] *** IcmMplxThread: exit thread ***

[Thr 139663697131856] JNCMIMsConnBrokenCallback: Message server connection broken

[Thr 139663697131856] IcmJncmConnBrokenCallback called

[Thr 139664033961808] Deactivated service 50004 for protocol P4 on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139664033961808] Deactivated service 50007 for protocol IIOP on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139664033961808] Deactivated service 50008 for protocol TELNET on host: "localhost"(only on this adapter) (timeout=60)

[Thr 139664033961808] *** ERROR => IcmServiceChanged: IcmJUpdateServiceList failed: -2 [icxxman_mt.c 4739]

[Thr 139663657183568] IcmWatchDogThread: Shutdown request received

[Thr 139663665576272] *** IcmMpiWatchDogThread: exit thread ***

[Thr 139663657183568] *** IcmWatchDogThread: exit thread ***

[Thr 139663698016592] *** IcmJSessCommThread: exit thread ***

[Thr 139664033961808] Removed service 50000 for protocol HTTP on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139664033961808] Removed service 50004 for protocol P4 on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139664033961808] Removed service 50007 for protocol IIOP on host: "ustrlxqa08.genre.com"(on all adapters) (timeout=60)

[Thr 139664033961808] Removed service 50008 for protocol TELNET on host: "localhost"(only on this adapter) (timeout=60)

[Thr 139664033961808] IcmJsfDetach: detach from Java Startup Framework

[Thr 139664033961808] *** ICM shutdown completed (pid: 23339) ***

[Former Member]

Hi Markus ,

Kindly find the logs of dev_jstart

-

---

trc file: "dev_jstart.new", trc level: 1, release: "720"

-

---

sysno 00

sid SCE

systemid 390 (AMD/Intel x86_64 with Linux)

relno 7200

patchlevel 0

patchno 43

intno 20020600

make multithreaded, Unicode, 64 bit, optimized

profile /usr/sap/SCE/SYS/profile/SCE_J00_ustrlxqa08

pid 15739

*

• ACTIVE TRACE LEVEL 1

• ACTIVE TRACE COMPONENTS All, egi

*

Tue Jun 14 08:59:12 2011

*

• trace logging activated, max size = 52428800 bytes, 2 versions

*

arguments :

arg[ 0] : jc.sapSCE_J00

arg[ 1] : pf=/usr/sap/SCE/SYS/profile/SCE_J00_ustrlxqa08

F

F Tue Jun 14 08:59:12 2011

F ********************************************************************************

F Java environment properties (/usr/sap/SCE/J00/work/jstart.jvm)

F root directory : /usr/sap/SCE/J00/exe/sapjvm_6

F vendor : SAP AG

F version : 1.6.0_24

F cpu : amd64

F java vm type : server

F java vm version : 6.1.026

F jvm library name : libjvm.so

F library path : /usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64/server:/usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64

F executable path : /usr/sap/SCE/J00/exe/sapjvm_6/bin

F ********************************************************************************

F SAP extensions : available

F ********************************************************************************

I [Thr 140288011753296] MtxInit: 30002 0 2

F [Thr 140288011753296] *** LOG => connected to Enqueue Server.

F [Thr 140288011753296] *** LOG => connected to Message Server.

F [Thr 140288011753296] *** LOG => Starting run level 1.

F ********************************************************************************

F Java process [deployment] properties:

F section name : deployment

F config file : /usr/sap/SCE/J00/exe/startup.properties

F node name : Deploy_offline

F home directory : /usr/sap/SCE/J00/j2ee/cluster

F shutdown timeout : 136000 ms

F exit timeout : 7000 ms

F debuggable : false

F debugger active : false

F ********************************************************************************

F [Thr 140288011753296] *** LOG => Instance state is "Deploying offline components" (STARTING @ 0, INACTIVE).

F [Thr 140288011753296] *** LOG => Starting nodes: runlevel 1.

F ********************************************************************************

F Starting process: /usr/sap/SCE/J00/exe/jstart

F arg[ 1] = -nodeId=0

F arg[ 2] = pf=/usr/sap/SCE/SYS/profile/SCE_J00_ustrlxqa08

F arg[ 3] = -hostvm

F arg[ 4] = -nodeName=deployment

F arg[ 5] = -file=/usr/sap/SCE/J00/exe/startup.properties

F arg[ 6] = -jvmFile=/usr/sap/SCE/J00/work/jstart.jvm

F arg[ 7] = -traceFile=/usr/sap/SCE/J00/work/dev_deployment

F arg[ 8] = -javaOutFile=/usr/sap/SCE/J00/work/jvm_deployment.out

F exePath : PATH=/usr/sap/SCE/J00/exe/sapjvm_6/bin:/usr/sap/SCE/J00/exe:/oracle/SCE/102_64/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib64/jvm/jre/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/usr/sap/SCE/SYS/exe/run:/home/sceadm:.

F libPath : LD_LIBRARY_PATH=/usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64/server:/usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64:/usr/sap/SCE/J00/j2ee/os_libs:/usr/sap/SCE/J00/exe:/usr/sap/SCE/SYS/exe/run:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/oracle/client/10x_64/instantclient

F stdout : /usr/sap/SCE/J00/work/std_deployment.out

F stderr : /usr/sap/SCE/J00/work/std_deployment.out

F console : no

F debugger: no

F nice : no

F ********************************************************************************

F Process Deploy_offline started with pid 15748

F ********************************************************************************

F [Thr 140288011753296] *** LOG => Process deployment started (pid 15748).

F

F [Thr 140288011753296] Tue Jun 14 08:59:15 2011

F [Thr 140288011753296] *** LOG => Process deployment stopping (pid 15748).

F [Thr 140288011753296] *** LOG => Signal 17 SIGCHLD.

F [Thr 140288011753296] *** LOG => Process deployment stopped (pid 15748).

F [Thr 140288011753296] *** LOG => Instance state is "Deploying offline components" (STOPPING @ 0, INACTIVE).

F [Thr 140288011753296] *** LOG => Run level 1 completed.

F [Thr 140288011753296] *** LOG => Starting run level 2.

F ********************************************************************************

F Java process [bootstrap] properties:

F section name : bootstrap

F config file : /usr/sap/SCE/J00/exe/startup.properties

F node name : Instance_bootstrap

F home directory : /usr/sap/SCE/J00/j2ee/cluster

F shutdown timeout : 136000 ms

F exit timeout : 7000 ms

F debuggable : false

F debugger active : false

F ********************************************************************************

F [Thr 140288011753296] *** LOG => Instance state is "Synchronizing binaries" (STARTING @ 0, INACTIVE).

F [Thr 140288011753296] *** LOG => Starting nodes: runlevel 2.

F ********************************************************************************

F Starting process: /usr/sap/SCE/J00/exe/jstart

F arg[ 1] = -nodeId=0

F arg[ 2] = pf=/usr/sap/SCE/SYS/profile/SCE_J00_ustrlxqa08

F arg[ 3] = -hostvm

F arg[ 4] = -nodeName=bootstrap

F arg[ 5] = -file=/usr/sap/SCE/J00/exe/startup.properties

F arg[ 6] = -jvmFile=/usr/sap/SCE/J00/work/jstart.jvm

F arg[ 7] = -traceFile=/usr/sap/SCE/J00/work/dev_bootstrap

F arg[ 8] = -javaOutFile=/usr/sap/SCE/J00/work/jvm_bootstrap.out

F exePath : PATH=/usr/sap/SCE/J00/exe/sapjvm_6/bin:/usr/sap/SCE/J00/exe:/oracle/SCE/102_64/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib64/jvm/jre/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/usr/sap/SCE/SYS/exe/run:/home/sceadm:.

F libPath : LD_LIBRARY_PATH=/usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64/server:/usr/sap/SCE/J00/exe/sapjvm_6/jre/lib/amd64:/usr/sap/SCE/J00/j2ee/os_libs:/usr/sap/SCE/J00/exe:/usr/sap/SCE/SYS/exe/run:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/oracle/client/10x_64/instantclient

F stdout : /usr/sap/SCE/J00/work/std_bootstrap.out

F stderr : /usr/sap/SCE/J00/work/std_bootstrap.out

F console : no

F debugger: no

F nice : no

F ********************************************************************************

F Process Instance_bootstrap started with pid 15813

F ********************************************************************************

F [Thr 140288011753296] *** LOG => Process bootstrap started (pid 15813).

F

F [Thr 140288011753296] Tue Jun 14 08:59:21 2011

F [Thr 140288011753296] *** LOG => Process bootstrap stopping (pid 15813).

F [Thr 140288011753296] *** LOG => Signal 17 SIGCHLD.

F [Thr 140288011753296] *** LOG => Process bootstrap stopped (pid 15813).

F [Thr 140288011753296] *** WARNING => Node bootstrap failed: result 1, exit code 503. [sfxxnode.hpp 1024]

F ********************************************************************************

F Native process [snapshot] properties:

F section name : snapshot

F config file : /usr/sap/SCE/J00/exe/startup.properties

F node name : snapshot

F home directory : /usr/sap/SCE/J00/work

F shutdown timeout : 122000 ms

F sf support : false

F ********************************************************************************

F ********************************************************************************

F Starting process: /usr/sap/SCE/J00/exe/sapcontrol

F arg[ 1] = -prot

F arg[ 2] = PIPE

F arg[ 3] = -nr

F arg[ 4] = 00

F arg[ 5] = -function

F arg[ 6] = CreateSnapshot

F arg[ 7] = Servercrash

F arg[ 8] = postmortem?node=bootstrap&exitcode=503

F arg[ 9] = 1

F arg[10] = 10000

F arg[11] = ""

F arg[12] = ""

F arg[13] = DEFAULT

F exePath : PATH=/usr/sap/SCE/J00/exe:/oracle/SCE/102_64/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib64/jvm/jre/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/usr/sap/SCE/SYS/exe/run:/home/sceadm:.

F libPath : LD_LIBRARY_PATH=/usr/sap/SCE/J00/exe:/usr/sap/SCE/SYS/exe/run:/usr/sap/SCE/SYS/exe/uc/linuxx86_64:/oracle/client/10x_64/instantclient

F stdout : /usr/sap/SCE/J00/work/std_snapshot.out

F stderr : /usr/sap/SCE/J00/work/std_snapshot.out

F console : no

F debugger: no

F nice : no

F ********************************************************************************

F Process snapshot started with pid 15899

F ********************************************************************************

F [Thr 140288011753296] *** LOG => Process snapshot started (pid 15899).

F [Thr 140288011753296] *** LOG => Process snapshot running (pid 15899).

F

F [Thr 140288011753296] Tue Jun 14 08:59:23 2011

F [Thr 140288011753296] *** LOG => SAP Start Service (pid 5160) connected.

F [Thr 140288011753296] *** LOG => SAP Start Service (pid 5160) disconnected.

F

F [Thr 140288011753296] Tue Jun 14 08:59:27 2011

F [Thr 140288011753296] *** LOG => Signal 17 SIGCHLD.

F [Thr 140288011753296] *** LOG => Process snapshot stopped (pid 15899).

F [Thr 140288011753296] *** LOG => Instance state is "Synchronizing binaries" (STOPPING @ 0, INACTIVE).

F [Thr 140288011753296] *** LOG => Run level 2 completed.

F [Thr 140288011753296] *** LOG => Instance state is "Some processes failed" (STOPPED @ 0, INACTIVE).

I [Thr 140288011753296] MPI: dynamic quotas disabled.

I [Thr 140288011753296] MPI init: pipes=4000 buffers=1279 reserved=383 quota=10%

F

F ********************************************************************************

F *** ERROR => Node 'bootstrap' failed with exit code 503.

F ***

F *** Please see section 'Failures in the 'synchronizing binaries' phase'

F *** in SAP Note 1316652 for additional information and trouble shooting advice.

F ********************************************************************************

F

F [Thr 140288011753296] *** LOG => exiting (exitcode 22002, retcode 1).

[Former Member]

Hi Markus ,

Kindly find the logs of dev_bootstrap :

J Tue Jun 14 08:59:17 2011

J [GC 4032K->1240K(9984K), 0.0153290 secs]

J [GC 5272K->3390K(14016K), 0.0164930 secs]

J [GC 11454K->8268K(16512K), 0.0259040 secs]

J [Full GCJ

J Tue Jun 14 08:59:18 2011

8268K->7147K(26688K), 0.0357710 secs]

J [GC 15211K->11914K(32384K), 0.0205870 secs]

F

F [Thr 140327935125840] Tue Jun 14 08:59:18 2011

F [Thr 140327935125840] *** LOG => State changed from 0 (Initial) to 2 (Starting framework).

F [Thr 140327935125840] *** LOG state real time: 2.043 CPU time: 0.890 sys, 2.210 usr

F [Thr 140327935125840] *** LOG total real time: 2.043 CPU time: 0.890 sys, 2.210 usr

F [Thr 140327935125840]

J [GC 25674K->18133K(34752K), 0.0256410 secs]

J [Full GC 18133K->13320K(45120K), 0.0588420 secs]

J

J Tue Jun 14 08:59:19 2011

J [GC 29448K->14546K(59712K), 0.0193400 secs]

J [GC 37522K->14498K(59776K), 0.0074410 secs]

J [GC 37474K->15314K(69952K), 0.0115520 secs]

J [GC 49746K->14866K(70848K), 0.0070150 secs]

J

J Tue Jun 14 08:59:20 2011

J [GC 49298K->15911K(78784K), 0.0121010 secs]

F

F [Thr 140327935125840] Tue Jun 14 08:59:21 2011

F [Thr 140327935125840] *** LOG => State changed from 2 (Starting framework) to 5 (Stopping).

F [Thr 140327935125840] *** LOG state real time: 3.387 CPU time: 0.250 sys, 3.650 usr

F [Thr 140327935125840] *** LOG total real time: 5.431 CPU time: 1.140 sys, 5.860 usr

F [Thr 140327935125840]

F

F [Thr 140324322568528] Tue Jun 14 08:59:21 2011

F [Thr 140324322568528] *** LOG => SfCJavaVm: exit hook is called. (rc = 503)

F

F Tue Jun 14 08:59:21 2011

F

F ********************************************************************************

F *** ERROR => Java node 'Instance_bootstrap' terminated with exit code 503.

F ***

F *** Please see section 'Java program exit codes'

F *** in SAP Note 1316652 for additional information and trouble shooting advice.

F ********************************************************************************

F

F [Thr 140328081934160] *** LOG => exiting (exitcode 503, retcode 1).

M [Thr 140328081934160] CCMS: CCMS Monitoring Cleanup finished successfully.

Kindly suggests where will be the problem .

[markus_doehr2]

What is the content of

/etc/sysctl.conf

Markus

[Former Member]

Hi Markus .

Thanks for the reply .

Below are the sysctl .conf logs .

1. Disable response to broadcasts.

2. You don't want yourself becoming a Smurf amplifier.

net.ipv4.icmp_echo_ignore_broadcasts = 1

1. enable route verification on all interfaces

net.ipv4.conf.all.rp_filter = 1

1. enable ipV6 forwarding

#net.ipv6.conf.all.forwarding = 1

1. increase the number of possible inotify(7) watches

fs.inotify.max_user_watches = 65536

1. avoid deleting secondary IPs on deleting the primary IP

net.ipv4.conf.default.promote_secondaries = 1

net.ipv4.conf.all.promote_secondaries = 1

1. Added for SAP on 2010-09-21 16:48:06 UTC

#kernel.shmmax = -1

1. Modified for SAP on 2011-05-25 05:40:15 UTC

#kernel.shmmax=11800422645

1. Modified for SAP on 2011-06-06 21:07:05 UTC

#kernel.shmmax = 21474836480

kernel.shmmax = 22548578304

kernel.sem = 1250 256000 100 8192

kernel.shmall = 1152921504606846720

vm.max_map_count = 300000

vm.hugetlb_shm_group = 1

~

Thanks

S SURESH

[Former Member]

Hi Markus ,

Wether this message was helpfull ?

Thanks

S SURESH

[markus_doehr2]

are you by accident using the same system numbers for both systems?

Markus