SOLUTION MANAGER UPGRADE - Error when shadow system Starting

[Former Member]

Hi All, We are upgrading our Solman system from 7.0 EHP 1 to SOLMAN 7.1. But the system is uable to start the Shadow

system. *** ERROR => DpHdlDeadWp: W0 (pid 12046) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12046) exited with exit code 255 *** ERROR => DpHdlDeadWp: W4 (pid 12050) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12050) exited with exit code 255 DpMBufHwIdSet: set Hardware-ID ***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c 1296] MBUF state ACTIVE DpWpBlksLow: max wp blocks in queue is 800 (80 %) MBUF component UP DpMsgProcess: 1 server in MBUF DpAppcBlksLow: max appc blocks in queue is 500 (50 %) *** ERROR => DpHdlDeadWp: W3 (pid 12049) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12049) exited with exit code 255 Sun Jan 8 21:50:33 2012 *** ERROR => DpHdlDeadWp: W2 (pid 12048) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12048) exited with exit code 255 *** ERROR => DpHdlDeadWp: W5 (pid 12051) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12051) exited with exit code 255 *** ERROR => DpHdlDeadWp: W6 (pid 12052) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12052) exited with exit code 255 *** ERROR => DpHdlDeadWp: W7 (pid 12053) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12053) exited with exit code 255 *** ERROR => DpHdlDeadWp: W8 (pid 12054) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12054) exited with exit code 255 *** ERROR => DpHdlDeadWp: W9 (pid 12055) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12055) exited with exit code 255 *** ERROR => DpHdlDeadWp: W10 (pid 12056) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12056) exited with exit code 255 *** ERROR => DpHdlDeadWp: W11 (pid 12057) died (severity=0, status=65280) [dpxxwp.c 1522] DpTraceWpStatus: child (pid=12057) exited with exit code 255 kill(12045,0) -> ESRCH: process died DpStartStopMsg: send stop message (myname is >solmanqazone_KSQ_04 <) DpStartStopMsg: Write AD_STARTSTOP message with type= 0, name=solmanqazone_KSQ_04 , sapsysnr= 4, hostname=solmanqazone AdGetSelfIdentRecord: > < AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0 AdCallRegisteredCvtToExt: opcode 60 AdCallRegisteredCvtToExt: opcode 60 call 102164870 AdCallRegisteredCvtToExt: opcode 60 exit rc=SAP_O_K AdCvtRecToExt: opcode 4 (AD_STARTSTOP), ser 0, ex 0, errno 0 AdCallRegisteredCvtToExt: opcode 4 AdCallRegisteredCvtToExt: opcode 4 call 102160ff8 AdCallRegisteredCvtToExt: opcode 4 exit rc=SAP_O_K DpConvertRequest: net size = 189 bytes NiBufIAlloc: malloc MSLIB-BUF, to 32110 bytes NiBufSend starting NiIWrite: hdl 17 sent data (wrt=562,pac=1,MESG_IO) MsINiWrite: sent 562 bytes MsISnd2: send msg (ms hdr/msg 110/452 bytes) to name -, type 4, key - DpStartStopMsg: stop msg sent NiBufIAlloc: malloc MSLIB-BUF, to 32110 bytes NiBufIAlloc: malloc NIBUF-IN, to 32110 bytes NiIRead: hdl 17 received data (rcd=274,pac=1,MESG_IO) NiBufIIn: NIBUF len=274 NiBufIIn: packet complete for hdl 17 NiBufSend starting NiIWrite: hdl 17 sent data (wrt=114,pac=1,MESG_IO) MsINiWrite: sent 114 bytes MsISnd2: send msg (ms hdr/msg 110/4 bytes) to name MSG_SERVER, type 0, key - MsSndName: MS_NOOP ok Send 4 bytes to MSG_SERVER NiBufIAlloc: malloc MSLIB-BUF, to 32110 bytes NiBufIAlloc: malloc NIBUF-IN, to 32110 bytes NiIRead: hdl 17 received data (rcd=114,pac=1,MESG_IO) NiBufIIn: NIBUF len=114 NiBufIIn: packet complete for hdl 17 NiBufReceive starting MsINiRead: received 114 bytes MsIReceive: received msg (ms hdr/msg 110/4 bytes), flag 3, from MSG_SERVER , typ 0, key - Received 4 bytes from MSG_SERVER Received opcode MS_NOOP from msg_server, reply MSOP_OK MsOpReceive: ok MsISendKeepalive : keepalive sent to message server NiIRead: hdl 17 recv would block (errno=EAGAIN) Sun Jan 8 21:52:21 2012 NiIPeek: peek for hdl 17 timed out (r; 1000ms) NiIRead: read for hdl 17 timed out (1000ms) DpHalt: no more messages from the message server DpHalt: sync with message server o.k. DpHalt: detach from message server ***LOG Q0M=> DpMsDetach, ms_detach () [dpxxdisp.c 12730] *** ERROR => e=28 semget(20465,1,2016) (28: No space left on device) [semux.c 498] *** ERROR => SemRq: Implicit SemInit failed. Key=65 [semux.c 1129] *** ERROR => Es2Cleanup: SemRq SEM_ES2_ADM (rc=1) [es2xx.c 1082] SemRq( 38, 1, -1) enter SemRq( 38, 1, -1) exit rtc=0, 0us SemRel( 38, 1 ) enter SemRel( 38, 1 ) exit rtc=0, 0us ShmDelete2( 73 ) DpHalt: cleanup event management DpHalt: cleanup shared memory/semaphores SemKeyPermission( 1 ) = 0740 (octal) SemKeyPermission( 1 ) = 0740 (octal) SemKeyPermission( 6 ) = 0740 (octal) SemKeyPermission( 6 ) = 0740 (octal) SemKeyPermission( 7 ) = 0740 (octal) SemKeyPermission( 8 ) = 0740 (octal) SemKeyPermission( 10 ) = 0740 (octal) SemKeyPermission( 11 ) = 0740 (octal) SemKeyPermission( 12 ) = 0740 (octal) SemKeyPermission( 13 ) = 0740 (octal) SemKeyPermission( 14 ) = 0740 (octal) SemKeyPermission( 15 ) = 0740 (octal) SemKeyPermission( 16 ) = 0740 (octal) SemKeyPermission( 17 ) = 0740 (octal) SemKeyPermission( 18 ) = 0740 (octal) SemKeyPermission( 19 ) = 0740 (octal) SemKeyPermission( 20 ) = 0740 (octal) SemKeyPermission( 21 ) = 0740 (octal) SemKeyPermission( 22 ) = 0740 (octal) *** ERROR => e=28 semget(20422,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 24 ) = 0740 (octal) *** ERROR => e=28 semget(20424,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 25 ) = 0740 (octal) *** ERROR => e=28 semget(20425,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 26 ) = 0740 (octal) *** ERROR => e=28 semget(20426,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 27 ) = 0740 (octal) *** ERROR => e=28 semget(20427,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 28 ) = 0740 (octal) *** ERROR => e=28 semget(20428,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 29 ) = 0740 (octal) *** ERROR => e=28 semget(20429,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 30 ) = 0740 (octal) *** ERROR => e=28 semget(20430,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 31 ) = 0740 (octal) *** ERROR => e=28 semget(20431,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 35 ) = 0740 (octal) *** ERROR => e=28 semget(20435,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 42 ) = 0740 (octal) *** ERROR => e=28 semget(20442,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 43 ) = 0740 (octal) *** ERROR => e=28 semget(20443,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 44 ) = 0740 (octal) *** ERROR => e=28 semget(20444,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 45 ) = 0740 (octal) *** ERROR => e=28 semget(20445,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 47 ) = 0740 (octal) *** ERROR => e=28 semget(20447,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 48 ) = 0740 (octal) *** ERROR => e=28 semget(20448,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 49 ) = 0740 (octal) *** ERROR => e=28 semget(20449,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 50 ) = 0740 (octal) *** ERROR => e=28 semget(20450,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 51 ) = 0740 (octal) *** ERROR => e=28 semget(20451,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 53 ) = 0740 (octal) *** ERROR => e=28 semget(20453,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 54 ) = 0740 (octal) *** ERROR => e=28 semget(20454,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 55 ) = 0740 (octal) *** ERROR => e=28 semget(20455,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 56 ) = 0740 (octal) *** ERROR => e=28 semget(20456,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 57 ) = 0740 (octal) *** ERROR => e=28 semget(20457,1,2016) (28: No space left on device) [semux.c 498] SemKeyPermission( 58 ) = 0740 (octal) DpCleanupSharedResources: MpiCleanup ShmCleanup( 62 ) ShmCreate( 62, 0, 2, 0xffffffff7fffea78 ) ShmProtect( 62, 3 ) ShmKeySharedMMU( 62 ) = 0 (octal) ShmProtect: shmat key 62 prot 3/0 done ShmCreate( 62, 0, 2, -> 0xfffffffe1d400000 ) MpiCleanup() -> MPI_OK DpCleanupSharedResources: removing Semaphore-Management DpCleanupSharedResources: removing request queue ShmCleanup( 31 ) ShmCreate( 31, 0, 2, 0xffffffff7fffea78 ) ShmProtect( 31, 3 ) ShmKeySharedMMU( 31 ) = 0 (octal) ShmProtect: shmat key 31 prot 3/0 done ShmCreate( 31, 0, 2, -> 0xffffffff24000000 ) DpCleanupSharedResources: ShmCleanup SHM_SYS_ADM_KEY ShmCleanup( 1 ) ShmCreate( 1, 0, 2, 0xffffffff7fffeb28 ) ShmProtect( 1, 3 ) ShmKeySharedMMU( 1 ) = 0 (octal) ShmProtect: shmat key 1 prot 3/0 done ShmCreate( 1, 0, 2, -> 0xffffffff78100000 ) DpCleanupSharedResources: ShmCleanup SHM_DP_ADM_KEY ShmCleanup( 2 ) ShmCreate( 2, 0, 2, 0xffffffff7fffeb28 ) ShmProtect( 2, 3 ) ShmKeySharedMMU( 2 ) = 0 (octal) ShmProtect: shmat key 2 prot 3/0 done ShmCreate( 2, 0, 2, -> 0xffffffff28800000 ) DpCleanupSharedResources: ShmCleanup SHM_DP_CA_KEY ShmCleanup( 3 ) ShmCreate( 3, 0, 2, 0xffffffff7fffeb28 ) ShmProtect( 3, 3 ) ShmKeySharedMMU( 3 ) = 0 (octal) ShmProtect: shmat key 3 prot 3/0 done ShmCreate( 3, 0, 2, -> 0xffffffff24800000 ) DpCleanupSharedResources: ShmCleanup SHM_PF_KEY ShmCleanup( 4 ) ShmCreate( 4, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 4 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_SAPSCSA_AREA_KEY ShmCleanup( 5 ) ShmCreate( 5, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 5 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_PXA_KEY ShmCleanup( 6 ) ShmCreate( 6, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 6 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_VB_ADM_KEY ShmCleanup( 7 ) ShmCreate( 7, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 7 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_PAGING_AREA_KEY ShmCleanup( 8 ) ShmCreate( 8, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 8 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_ROLL_AREA_KEY ShmCleanup( 9 ) ShmCreate( 9, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 9 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_CALI_BUFFER ShmCleanup( 11 ) ShmCreate( 11, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 11, 0, 2, 0xffffffff7fffe9e0 ) DpCleanupSharedResources: ShmCleanup SHM_CCC_KEY ShmCleanup( 12 ) ShmCreate( 12, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 12, 0, 2, 0xffffffff7fffe9e0 ) DpCleanupSharedResources: ShmCleanup SHM_ALERT_AREA_KEY ShmCleanup( 13 ) ShmCreate( 13, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 13, 0, 2, 0xffffffff7fffe9e0 ) DpCleanupSharedResources: ShmCleanup SHM_PRES_BUF ShmCleanup( 14 ) ShmCreate( 14, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 14, 0, 2, 0xffffffff7fffe9e0 ) DpCleanupSharedResources: ShmCleanup DVS_SHM_KEY ShmCleanup( 15 ) ShmCreate( 15, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 15, 0, 2, 0xffffffff7fffe9e0 ) DpCleanupSharedResources: ShmCleanup SHM_SEM_MONI_BUFFER ShmCleanup( 16 ) ShmCreate( 16, 0, 2, 0xffffffff7fffeb28 ) ShmProtect( 16, 3 ) ShmCreate( 16, 0, 2, -> 0xffffffff29004000 ) ShmDelete2( 10 ) Sun Jan 8 21:52:22 2012 DpCleanupSharedResources: ShmCleanup SHM_ROLL_ADM_KEY ShmCleanup( 17 ) ShmCreate( 17, 0, 2, 0xffffffff7fffeb28 ) ShmGet( 10, 17, 0, 2, 0xffffffff7fffe9e0 ) ShmCreate( 10, 0, 2, 0xffffffff7fffe828 ) ShmKeyPermission( 10 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_PAGING_ADM_KEY ShmCleanup( 18 ) ShmCreate( 18, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 18 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_DB_TBUFF ShmCleanup( 19 ) ShmProtect( 52, 3 ) ShmKeySharedMMU( 52 ) = 0 (octal) ShmProtect: shmat key 52 prot 3/0 done ShmCreate( 52, 0, 2, -> 0xffffffff77600000 ) DpCleanupSharedResources: ShmCleanup SHM_ES_ADM_T ShmCleanup( 53 ) ShmCreate( 53, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 53 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_DB_OBJ_BUFFER ShmCleanup( 54 ) ShmCreate( 54, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 54 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_RSPO_LOCAL_KEY ShmCleanup( 55 ) ShmCreate( 55, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 55 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_PF_AS_KEY ShmCleanup( 56 ) ShmCreate( 56, 0, 2, 0xffffffff7fffeb28 ) ShmKeyPermission( 56 ) = 0740 (octal) DpCleanupSharedResources: ShmCleanup SHM_PROFILE ShmCleanup( 57 ) ShmCreate( 57, 0, 2, 0xffffffff7fffeb28 ) ShmProtect( 57, 3 ) ShmKeySharedMMU( 57 ) = 0 (octal) ShmProtect: shmat key 57 prot 3/0 done ShmCreate( 57, 0, 2, -> 0xffffffff75700000 ) DpCleanupSharedResources: ShmCleanup SHM_ENQID_KEY ShmCleanup( 58 )

Regards, VIjay K.G

[mateus_pedroso]

Hi Vijay

This issue isn't related with size of filesystem or partitioning. This issue is related with memory parameters.

Check these notes below and check if you set the correct parameters.

425207 - SAP memory management, current parameter ranges

146289 - Parameter Recommendations for 64-Bit SAP Kernel

If you have Solaris see notes 724713 & 1437105.

[Former Member]

Hi,

From logs:

28: No space left on device

Check the disk space. If linux, use df -h command. Windows its easier

Check if any drive / file system is 100% used up.

Regards,

Srikishan