Crash of MAXDb 7.7.06 - is it possible to restore?

[Former Member]

Hi all,

could anybody help. Our DB was crashed. Restoring to Online from Admin - unsuccessful.

We attempted to repair indexes in Admin state with command:

CHECK DATA (from database Studio).

, but get message:

SQL error

-9407,System error: unexpected error

100,Check database finished unsuccessfully

5,Mark index as not accessible

Servertask Info: because bd01CheckFile failed

Job 0 (Check Data) [executing] WaitingT79 Result=OK

bd01CheckFile failed, Error code 6433 "system_error"

3,Check index failed

105,Damaged B*tree structure found

Also we did a copy of crashed DB, restored it to another instance and attempted to Online. The same result.

Is it possible to restore DB (with or without rollback of the last operation)???

Now we are trying db_execute CHECK DATA EXTENDED WITH UPDATE (still running)

Thanks in advance

[Former Member]

Hi again,

sql_recreateindex could executed just in Online state (or maybe I am wrong??)

regards

[lbreddemann]

> sql_recreateindex could executed just in Online state (or maybe I am wrong??)

Yep - so what?

Of course it can only be executed in online state, because the command is just a convinient way to write

DROP INDEX ...

CREATE INDEX...

regards,

Lars

[Former Member]

I wrote: we could not turn db state to ONLINE (db_online). THAT IS THE PROBLEM. message is:

Runtime environment error

4,connection broken

So we asked about the possibility to repair indexes in ADMIN state.

Thanks

[lbreddemann]

>

> I wrote: we could not turn db state to ONLINE (db_online). THAT IS THE PROBLEM. message is:

>

> Runtime environment error

> 4,connection broken

>

> So we asked about the possibility to repair indexes in ADMIN state.

You're sure that the BAD INDEX is the problem here?

If you're a SAP customer, you should open a support message for this now.

If not, I'd need to see the part of the KNLSMG file (in readable form, please!) when the crash occurs.

As a shot into the blue you may try to set the UseSystemTrigger Parameter to NO and disable the automatic collection of statistics before starting the database to ONLINE mode.

Then you should be able to get the DB ONLINE without a crash.

regards,

Lars

[Former Member]

Thanks, Lars, for Your advice but it didn't help.

We set UseSystemTrigger Parameter to NO, the automatic collection of statistics was disabled. We got the same message.

File KNLMSG is realy big. I have two

1-crash extract (just first few messages from crash date)

2-set to Online after UseSystemTrigger Parameter=NO (just last few messages)

1:

-

---

<MSGL _NO="144307" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="53133" _COMP="RTEKERN" _TEXT="Second exclusive request on same page">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="DATACACH"

/>

</MSG>

</MSGL>

<MSGL _NO="144308" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="53133" _COMP="RTEKERN" _TEXT="PAGENO: 7295302">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="DATACACH"

/>

</MSG>

</MSGL>

<MSGL _NO="144309" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="53133" _COMP="RTEKERN" _TEXT="CBLOCKPTR: 0x7fef9639c00">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="DATACACH"

/>

</MSG>

</MSGL>

<MSGL _NO="144310" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="53000" _COMP="RTEKERN" _TEXT="0703000000000000039A000000000000">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="B*TREE"

/>

</MSG>

</MSGL>

<MSGL _NO="144311" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="53000" _COMP="RTEKERN" _TEXT="Index Root 6656239">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="B*TREE"

/>

</MSG>

</MSGL>

<MSGL _NO="144312" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="18196" _COMP="RTEKERN" TEXT="vabort:Emergency Shutdown, KernelAdministration.cpp: 704">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-23 08:42:06.700"

_LABEL="DBCRASH "

/>

</MSG>

</MSGL>

<MSGL _NO="144313" _PROCESS="5072" _THREAD="0x10D8" _TASK="189" _TIME="2010-04-23 08:42:06.700">

<MSG _NO="1" _TYPE="Error" _ID="3" _COMP="Admin" _TEXT="Database state: OFFLINE">

<MSG_ARGS

FILE="KernelAdministration.cpp"

_LINE="704"

_TIME="2010-04-23 08:42:06.700"

_MESSAGEVERSION="1"

/>

</MSG>

<MSG _NO="2" _TYPE="Error" _ID="2" _COMP="Admin" _TEXT="A fatal error caused EMERGENCY SHUTDOWN. $ERRORTEXT$">

<MSG_ARGS

_FILE="vgg999.cpp"

_LINE="62"

_TIME="2010-04-23 08:42:06.700"

_MESSAGEVERSION="1"

ERRORTEXT="core is written"

/>

</MSG>

-

---

2:

-

---

<MSGL _NO="154521" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:16.621">

<MSG _NO="1" _TYPE="Error" _ID="0" _COMP="RTEKERN" _TEXT=" SFrame: IP: 0x0000000077a2a1ed (0x0000000077a29f70+0x27d) FP: 0x000000000bbeffd0 SP: 0x000000000bbeffb0">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-28 14:34:16.621"

_LABEL="DIAG"

/>

</MSG>

</MSGL>

<MSGL _NO="154522" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:16.621">

<MSG _NO="1" _TYPE="Error" _ID="0" _COMP="RTEKERN" _TEXT=" Params: 0x0, 0x0, 0x0, 0x0">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-28 14:34:16.621"

_LABEL="DIAG"

/>

</MSG>

</MSGL>

<MSGL _NO="154523" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:16.621">

<MSG _NO="1" _TYPE="Error" _ID="0" _COMP="RTEKERN" _TEXT=" Module: C:\Windows\system32\kernel32.dll">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-28 14:34:16.621"

_LABEL="DIAG"

/>

</MSG>

</MSGL>

<MSGL _NO="154524" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _TYPE="Error" _ID="0" _COMP="RTEKERN" _TEXT="Mini dump file &apos;KnlMini.dmp&apos; written!">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="709"

_TIME="2010-04-28 14:34:18.807"

_LABEL="DIAG"

/>

</MSG>

</MSGL>

<MSGL _NO="154525" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _TYPE="Error" _ID="7" _COMP="Messages" _TEXT="Begin of dump of registered messages">

<MSG_ARGS

FILE="MsgList.cpp"

_LINE="3539"

_TIME="2010-04-28 14:34:18.807"

_MESSAGEVERSION="1"

/>

</MSG>

</MSGL>

<MSGL _NO="154526" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _ID="13951" _COMP="RTEHSS" _TEXT="No hot standby node configured -&gt; No HotStandby configuration">

<MSG_ARGS

FILE="RTEHSSKernelInterface.cpp"

_LINE="840"

_TIME="2010-04-28 14:20:50.689"

/>

</MSG>

</MSGL>

<MSGL _NO="154527" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _TYPE="Error" _ID="2" _COMP="Admin" _TEXT="A fatal error caused EMERGENCY SHUTDOWN. $ERRORTEXT$">

<MSG_ARGS

_FILE="vgg999.cpp"

_LINE="62"

_TIME="2010-04-28 14:34:15.903"

_MESSAGEVERSION="1"

ERRORTEXT="core is written"

/>

</MSG>

</MSGL>

<MSGL _NO="154528" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _TYPE="Error" _ID="3" _COMP="Admin" _TEXT="Database state: OFFLINE">

<MSG_ARGS

FILE="KernelAdministration.cpp"

_LINE="704"

_TIME="2010-04-28 14:34:15.903"

_MESSAGEVERSION="1"

/>

</MSG>

<MSG _NO="2" _TYPE="Error" _ID="2" _COMP="Admin" _TEXT="A fatal error caused EMERGENCY SHUTDOWN. $ERRORTEXT$">

<MSG_ARGS

_FILE="vgg999.cpp"

_LINE="62"

_TIME="2010-04-28 14:34:15.903"

_MESSAGEVERSION="1"

ERRORTEXT="core is written"

/>

</MSG>

</MSGL>

<MSGL _NO="154529" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

</MSGL>

<MSGL _NO="154530" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:18.807">

<MSG _NO="1" _TYPE="Error" _ID="8" _COMP="Messages" _TEXT="End of the message list registry dump">

<MSG_ARGS

FILE="MsgList.cpp"

_LINE="3567"

_TIME="2010-04-28 14:34:18.807"

_MESSAGEVERSION="1"

/>

</MSG>

</MSGL>

<MSGL _NO="154553" _PROCESS="5036" _THREAD="0x628" _TIME="2010-04-28 14:34:31.225">

<MSG _NO="1" _ID="28" _COMP="MsgOutput" TEXT="__ Stopping GMT 2010-04-28 11:34:31 7.7.06 Build 009-123-202-944">

<MSG_ARGS

FILE="MsgOutputKnlDiag.cpp"

_LINE="769"

_TIME="2010-04-28 14:34:31.225"

/>

</MSG>

-

---

[lbreddemann]

>

> Thanks, Lars, for Your advice but it didn't help.

> We set UseSystemTrigger Parameter to NO, the automatic collection of statistics was disabled. We got the same message.

>

> File KNLMSG is realy big. I have two

> 1-crash extract (just first few messages from crash date)

> 2-set to Online after UseSystemTrigger Parameter=NO (just last few messages)

>

Sorry - but without the log in human readable format I won't look into this.

Use protconv or one of the GUI tools to reformat the pseudo-XML files to plain text files.

And once again: if you're a SAP customer, do open a support message for this!

regards,

Lars

[lbreddemann]

>

> Hi all,

>

> could anybody help. Our DB was crashed. Restoring to Online from Admin - unsuccessful.

>

> We attempted to repair indexes in Admin state with command:

> CHECK DATA (from database Studio).

>

> , but get message:

> SQL error

> -9407,System error: unexpected error

> 100,Check database finished unsuccessfully

> 5,Mark index as not accessible

> Servertask Info: because bd01CheckFile failed

> Job 0 (Check Data) [executing] WaitingT79 Result=OK

> bd01CheckFile failed, Error code 6433 "system_error"

> 3,Check index failed

> 105,Damaged B*tree structure found

>

> Also we did a copy of crashed DB, restored it to another instance and attempted to Online. The same result.

>

> Is it possible to restore DB (with or without rollback of the last operation)???

>

> Now we are trying db_execute CHECK DATA EXTENDED WITH UPDATE (still running)

Well, looks like NO RECOVERY at all is required here.

Simply figure out what index is affected (look for BAD INDEXES in DB50, DBMGUI, DB Studio or DBMCLI) and rebuild them via

sql_recreateindex

That should do the trick.

Regards,

Lars

[Former Member]

You can perform a restore and a point in time recovery just before you executed this operation.

Kind regards,

Mark

[lbreddemann]

>

> You can perform a restore and a point in time recovery just before you executed this operation.

Sorry, but a PIT is the worst thing to do in that case!

Is this a user error? A mistake in data processing?

Nope.

It's a corruption. That's it.

It's not something that would be there again, when you recover the database.

And what about all the lost business transactions?

Any idea how to efficiently handle the business data inconsistencies you create with a PIT?

PLEASE: Never (ever!) recommend to do a point in time recovery when it's not 100% sure to be the only solution and what to do about the inconsistent business data afterwards.

regards,

Lars

[Former Member]

Hello Lars,

I totally agree, but he asked if it is possible to do so.. Which in fact is.

If course, this should be the very last thing to do.

Kind regards,

Mark