Help to analyze a DUMP: SYSTEM_CORE_DUMPED

[ganimede_dignan]

Hi,

during lunch time we have several dump with SYSTEM_CORE_DUMPED, standard report as RSBTCRTE, SAPLSGUI, SAPLSPIAGENTCW and an other custom report.

Last error logged in SAP kernel

Component............ "Taskhandler"

Place................ "SAP-Server host_SID_00 on host host (wp 51)"

Version.............. 1

Error code........... 11

Error text........... "ThSigHandler: signal"

Description.......... " "

System call.......... " "

Module............... "thxxhead.c"

Line................. 10851

The error reported by the operating system is:

Error number..... " "

Error text....... " "

Have you got any advice? dev file there insn't other smart information.

It's a Solaris 10 box on SPARC platform with Oracle 10.2.0.4, ECC 6.0, Kernel

Regards.

Thank you for collaboration.

[former_member189546]

Hello,

You seem to have signal 11 error from the os.

Check memory parameters

146528 "Configuration of R/3 on hosts with much RAM".

Often a full kernel patch upgrade clears this. disp+work and dbsl.

Note 19466

regards,

John Feely

[ganimede_dignan]

.

[former_member126060]

Along with what Marcus wrote, core dumps are due to the system binaries or the database libraries.

so just in case you are not working with the latest binaries updating kernel and database library will solve your problem.

- Niraj

[ganimede_dignan]

It's a 239 patch level kernel 7.00 Unicode and DBLS is 205 patch

[markus_doehr2]

> during lunch time we have several dump with SYSTEM_CORE_DUMPED, standard report as RSBTCRTE, SAPLSGUI, SAPLSPIAGENTCW and an other custom report.

Coredumps are caused by unhandled exception in the ABAP runtime.

You can find the stack trace in the "analysis" section of the dump in ST22, there you see the C/C++ stack trace and you may see which library (e. g. the database client, the database interface etc.) is causing the problem.

Markus

[ganimede_dignan]

>You can find the stack trace in the "analysis" section of the dump in ST22, there you see the C/C++ stack trace and you may >see which library (e. g. the database client, the database interface etc.) is causing the problem.

Already attached...

Last error logged in SAP kernel

Component............ "Taskhandler"

Place................ "SAP-Server host_SID_00 on host host (wp 51)"

Version.............. 1

Error code........... 11

Error text........... "ThSigHandler: signal"

Description.......... " "

System call.......... " "

Module............... "thxxhead.c"

Line................. 10851

The error reported by the operating system is:

Error number..... " "

Error text....... " "

[ganimede_dignan]

M *****************************************************************************

M *

M * LOCATION SAP-Server host_SID_00 on host host (wp 51)

M * ERROR ThSigHandler: signal

M *

M * TIME Tue Jul 13 13:04:06 2010

M * RELEASE 700

M * COMPONENT Taskhandler

M * VERSION 1

M * RC 11

M * MODULE thxxhead.c

M * LINE 10851

M * COUNTER 1298

M *

M *****************************************************************************

M

M PfStatDisconnect: disconnect statistics

M Entering TH_CALLHOOKS

M ThCallHooks: call hook >SAP-Trace buffer write< for event BEFORE_DUMP

M TrThHookFunc: called for WP dump

M TrStopTrace: switch off Trace for this WP

M TrFlushTrace: flush the trace buffer to file

M ThGetTransId2: got trans id (spa) >4C3CDB0747D5421EE10000000A0205C9< (32)

M TrWriteBuffer: initrec: 0

M TrSaveFile: file can be kept (9732100/16777216 Bytes)

M TrIResetErrorAction: reset all errors for action 0

M TrWriteBuffer: 224 Bytes for block buffer

M TrBuffFlush: wrote 23420 Bytes of buffer to file

M TrWriteBuffer: 80 Bytes of block trailer

M TrIResetErrorAction: reset all errors for action 1

M TrFiICheckIndex: check index SAP-Trace tracefile list at 0xfffffff9c05153b8

M TrFiICheckIndex: check record in slot 0

M TrFiICheckIndex: check record in slot 1

M TrFiICheckIndex: check record in slot 2

M TrFiICheckIndex: check record in slot 3

M TrFiICheckIndex: check record in slot 4

M TrFiICheckIndex: check record in slot 5

M TrFiICheckIndex: check record in slot 6

M TrFiICheckIndex: check record in slot 7

M TrFiICheckIndex: check record in slot 8

M TrFiICheckIndex: check record in slot 9

M TrFiICheckIndex: check index SAP-Trace tracefile list at 0xfffffff9c05153b8

M TrFiICheckIndex: check record in slot 0

M TrFiICheckIndex: check record in slot 1

M TrFiICheckIndex: check record in slot 2

M TrFiICheckIndex: check record in slot 3

M TrFiICheckIndex: check record in slot 4

M TrFiICheckIndex: check record in slot 5

M TrFiICheckIndex: check record in slot 6

M TrFiICheckIndex: check record in slot 7

M TrFiICheckIndex: check record in slot 8

M TrFiICheckIndex: check record in slot 9

M TrFiICheckIndex: check index SAP-Trace tracefile list at 0xfffffff9c05153b8

M TrFiICheckIndex: check record in slot 0

M TrFiICheckIndex: check record in slot 1

M TrFiICheckIndex: check record in slot 2

M TrFiICheckIndex: check record in slot 3

M TrFiICheckIndex: check record in slot 4

M TrFiICheckIndex: check record in slot 5

M TrFiICheckIndex: check record in slot 6

M TrFiICheckIndex: check record in slot 7

M TrFiICheckIndex: check record in slot 8

M TrFiICheckIndex: check record in slot 9

M ThCallHooks: call hook >ThrSaveSPAFields< for event BEFORE_DUMP

M ThrSaveSPAFields: save spa fields

M ThrSaveSPAFields: not an update task, no update info saved

M ThrSaveSPAFields: save field zttabtckey (RSEOUT00_XML 13020302 %NEWSTEP IT 020Y) into thr_wpadm

M ThCallHooks: call hook >BtcCallLgCl< for event BEFORE_DUMP

M BtcCallLgCl: call function BtcLgCl

M SosISearchAnchor: anchorArray allocated but no value stored yet

M ThGetTransId2: got trans id (spa) >4C3CDB0747D5421EE10000000A0205C9< (32)

M Tue Jul 13 13:04:07 2010

M Entering ThSetStatError

M ThCallDbBreak: use db_sqlbreak

B db_sqlbreak() = 1

M ThIErrHandle: don't try rollback again

M ThIErrHandle: call ThrCoreInfo

A TH VERBOSE LEVEL FULL

M SigIRegisterRoutine: handler for signal 11 installed (SigIGenAction)

M SigIRegisterRoutine: handler for signal 10 installed (SigIGenAction)

M SigIRegisterHook: registered hook >ab_catch_dumperror< for signal 11

M SigIRegisterHook: registered hook >ab_catch_dumperror< for signal 10

A ** RABAX: level LEV_RX_PXA_RELEASE_MTX entered.

A ** RABAX: level LEV_RX_PXA_RELEASE_MTX completed.

A ** RABAX: level LEV_RX_COVERAGE_ANALYSER entered.

A ** RABAX: level LEV_RX_COVERAGE_ANALYSER completed.

A ** RABAX: level LEV_RX_SAVE_SHMLOCKS entered.

A ** RABAX: level LEV_RX_SAVE_SHMLOCKS completed.

A ** RABAX: level LEV_RX_RESET_SHMLOCKS entered.

A ** RABAX: level LEV_RX_RESET_SHMLOCKS completed.

A ** RABAX: level LEV_RX_ROLLBACK entered.

A ** RABAX: level LEV_RX_ROLLBACK completed.

A ** RABAX: level LEV_RX_DB_ALIVE entered.

A ** RABAX: level LEV_RX_DB_ALIVE completed.

A ** RABAX: level LEV_RX_HOOKS entered.

A ** RABAX: level LEV_RX_HOOKS completed.

A ** RABAX: level LEV_RX_STANDARD entered.

A ** RABAX: level LEV_RX_STANDARD completed.

A ** RABAX: level LEV_RX_STOR_VALUES entered.

A ** RABAX: level LEV_RX_STOR_VALUES completed.

A ** RABAX: level LEV_RX_C_STACK entered.

A Tue Jul 13 13:04:08 2010

A ** RABAX: level LEV_RX_C_STACK completed.

A ** RABAX: level LEV_RX_MEMO_CHECK entered.

A ** RABAX: level LEV_RX_MEMO_CHECK completed.

A ** RABAX: level LEV_RX_AFTER_MEMO_CHECK entered.

A ** RABAX: level LEV_RX_AFTER_MEMO_CHECK completed.

A ** RABAX: level LEV_RX_INTERFACES entered.

A ** RABAX: level LEV_RX_INTERFACES completed.

A ** RABAX: level LEV_RX_GET_MESS entered.

A ** RABAX: level LEV_RX_GET_MESS completed.

A ** RABAX: level LEV_RX_INIT_SNAP entered.

A ** RABAX: level LEV_RX_INIT_SNAP completed.

A ** RABAX: level LEV_RX_WRITE_SYSLOG entered.

A ** RABAX: level LEV_RX_WRITE_SYSLOG completed.

A ** RABAX: level LEV_RX_WRITE_SNAP_BEG entered.

A ** RABAX: level LEV_RX_WRITE_SNAP_BEG completed.

A ** RABAX: level LEV_RX_WRITE_SNAP entered.

M ThGetTransId2: got trans id (spa) >4C3CDB0747D5421EE10000000A0205C9< (32)

A ATRA-ERROR: RSBTCRTE 807: RABAX occurred.

A ** RABAX: level LEV_SN_END completed.

A ** RABAX: level LEV_RX_WRITE_SNAP_END entered.

A ** RABAX: level LEV_RX_WRITE_SNAP_END completed.

A ** RABAX: level LEV_RX_SET_ALERT entered.

A ** RABAX: level LEV_RX_SET_ALERT completed.

A ** RABAX: level LEV_RX_COMMIT entered.

A ** RABAX: level LEV_RX_COMMIT completed.

A ** RABAX: level LEV_RX_SNAP_SYSLOG entered.

A ** RABAX: level LEV_RX_SNAP_SYSLOG completed.

A ** RABAX: level LEV_RX_RESET_PROGS entered.

A ** RABAX: level LEV_RX_RESET_PROGS completed.

A ** RABAX: level LEV_RX_STDERR entered.

A ** RABAX: level LEV_RX_STDERR completed.

A ** RABAX: level LEV_RX_RFC_ERROR entered.

A ** RABAX: level LEV_RX_RFC_ERROR completed.

A ** RABAX: level LEV_RX_RFC_CLOSE entered.

A ** RABAX: level LEV_RX_RFC_CLOSE completed.

A ** RABAX: level LEV_RX_IMC_ERROR entered.

M SosISearchAnchor: anchorArray allocated but no value stored yet

A ** RABAX: level LEV_RX_IMC_ERROR completed.

A ** RABAX: level LEV_RX_DATASET_CLOSE entered.

A ** RABAX: level LEV_RX_DATASET_CLOSE completed.

A ** RABAX: level LEV_RX_ERROR_SAVE entered.

A ** RABAX: level LEV_RX_ERROR_SAVE completed.

A ** RABAX: level LEV_RX_ERROR_TPDA entered.

A ** RABAX: level LEV_RX_ERROR_TPDA completed.

A ** RABAX: level LEV_RX_PXA_RELEASE_RUDI entered.

A ** RABAX: level LEV_RX_PXA_RELEASE_RUDI completed.

A ** RABAX: level LEV_RX_LIVE_CACHE_CLEANUP entered.

M RmCleanUpResources3: hdr/tbl/ext_tbl/free/keep = 0xffffffff68196f50/(nil)/0/0/1

M RmCleanUpResources3: no resources registered

A ** RABAX: level LEV_RX_LIVE_CACHE_CLEANUP completed.

A ** RABAX: level LEV_RX_END entered.

A ** RABAX: level LEV_RX_END completed.

M SigIRegisterHook: cleared hook for signal 11

M SigIRegisterHook: cleared hook for signal 10

A ** RABAX: end RX_RETURN

M ThIErrHandle: ThrCoreInfo o.k.

M ThIErrHandle: Entering ThReplyToMsg

M ThIErrHandle: Entering ThErrHdlUser

M ThErrHdlUser: set th_errno (11)

M ThErrHdlUser: save context

M ThSaveContext: roll out

M ThRollOut: roll out T51/U52/M0/I1 (level=7, short_roll_out=0)

M ThRollOut: call rrol_out (1)

M ThRollOut: act roll state = DP_ROLLED_OUT

M ThRollOut: roll level <> 0, don't call ab_rollout

M ThRollOut: full roll out of U52 M0 I1 (Level 7) ok

M ThEmContextDetach2: detach T51/M0 from em memory (em_hdl=412, force=0)

M ThEmContextDetach2: reset local em info

B Disconnecting from ALL connections:

B Wp Hdl ConName ConId ConState TX HC PRM RCT FRC TIM MAX OPT Date Time DBHost

B 051 000 R/3 000000000 ACTIVE YES NO YES NO NO 000 255 255 20100713 130235 mnsun201

C Disconnecting from connection 0 ...

C Closing user session (con_hdl=0,svchp=0x1076fe488,usrhp=0x1076ee5d8)

C Detaching from DB Server (con_hdl=0,svchp=0x1076fe488,srvhp=0x107701458)

C Now I'm disconnected from ORACLE

B Disconnected from connection 0

B statistics db_con_commit (com_total=35, com_tx=18)

B statistics db_con_rollback (roll_total=0, roll_tx=0)

M ***LOG Q02=> wp_halt, WPStop (Workproc51 16926) [dpuxtool.c 269]

[markus_doehr2]

The section to check is

"active calls in the SAP kernel" in ST22, this is a C stack trace looking like

(CTrcStack2+0x82)[0x6c99e2]
(_Z16rabax_CStackSavev+0xe2)[0xb829c2]
(ab_rabax+0x4485)[0xb8b0e5]
(_Z13tsvAppendBodyP4TABHjPjh+0x31b)[0x93a2eb]
(ab_tnewNoIdxUpd+0x3e)[0x93ceee]
(_Z19cloneObjectDeepDataPK7AB_DATAPvPKvP18CloneObjectHashAdm+0x375)[0x95e2d5]
(_Z19cloneObjectDeepDataPK7AB_DATAPvPKvP18CloneObjectHashAdm+0x222)[0x95e182]
(_Z17cloneObjectObjectP6HeadIdPKS_P18CloneObjectHashAdm+0x22a)[0x95de7a]
(_Z14ab_cloneObjectP6HeadIdjPKS_+0x9f5)[0x96f805]
(_Z29ab_ShmAttachUpdateWithVersionPKtjS0_S0_jS0_14AbShmWriteModePiPK14AbShmAreaAtt_SPjPmS6_S6_PvP
(ab_ShmkmAttachUpdate70+0x35c)[0xacff6c]
(_Z8ab_jmethv+0x1a6)[0x9c2f36]
(_Z8ab_extriv+0xd51)[0x8b2bc1]
(_Z9ab_xeventPKt+0x2b5)[0xa1a625]
(ab_dstep+0x1be)[0xba110e]
(dynpmcal+0x799)[0x72f059]
(dynppai0+0x785)[0x730645]
(dynprctl+0x476)[0x731576]
(dynpen00+0x60c)[0x722f5c]
(TskhLoop+0x11cb)[0x54f31b]
(ThStart+0x1b1)[0x55c471]
(DpMain+0x228)[0x4bf258]

Markus

[ganimede_dignan]

Hi,

thank you... but what is it?

Regards.

[markus_doehr2]

> thank you... but what is it?

This is the C/C++ stacktrace, with that information one can see where the dump occurs (in the ABAP kernel, in the database interface library, in the database client etc).

Without that information everything here is just wild guessing.

Markus

[ganimede_dignan]

A collegue inform me that, more or less, him has start some traces on this system.

Regards.