Problem getting db_online state after 100% full log volume

[Former Member]

Hello,

Because of a 100% filled up log volume this database went down, and cant be started up again.

cicrp:crpadm 54> dbmcli -U c db_online

ERR

-24895,ERR_SHUTDOWN: shutdown of database occured

-71,connection broken server state 6

cicrp:crpadm 55>

From "knldiag.err":

2006-01-22 02:15:26 --- Starting GMT 2006-01-22 01:15:26 7.5.0 Build 015-121-074-67

5

2006-01-30 14:13:43 --- Starting GMT 2006-01-30 13:13:43 7.5.0 Build 015-121-074-67

5

2006-01-30 14:16:20 --- Starting GMT 2006-01-30 13:16:20 7.5.0 Build 015-121-074-67

5

2006-02-02 01:07:03 12950 ERR 11000 devio write error (fd = 25): (v) PNO 208172 Size 4 : Result 24576 (Interrupted sys

tem call)

2006-02-02 01:07:03 12952 ERR 11000 devio write error (fd = 28): (v) PNO 208172 Size 4 : Result 24576 (Interrupted sys

tem call)

2006-02-02 01:07:03 12931 ERR 11000 vectorio SYSTEM ERROR on devno 5

2006-02-02 01:07:03 12931 ERR 11000 vectorio SYSTEM ERROR on devno 6

2006-02-02 01:07:03 12931 ERR 15 IOMan Access error on DualLog volume 1 blockno 208172: Error during seek or write

2006-02-02 01:07:03 12931 ERR 15 IOMan Access error on DualLog volume 1 blockno 208172:

Ô&£@@

2006-02-02 01:07:03 12931 ERR 17 IOMan EMERGENCY SHUTDOWN

2006-02-02 01:07:03 12931 ERR 8 Admin ERROR '' CAUSED EMERGENCY SHUTDOWN

2006-02-02 01:08:39 --- Starting GMT 2006-02-02 00:08:39 7.5.0 Build 015-121-074-67

5

2006-02-02 01:08:57 30029 ERR 11000 devio write error (fd = 25): (v) PNO 1 Size 1 : No space left on device

2006-02-02 01:08:57 30031 ERR 11000 devio write error (fd = 28): (v) PNO 1 Size 1 : No space left on device

2006-02-02 01:08:57 30012 ERR 11000 vectorio SYSTEM ERROR on devno 5

2006-02-02 01:08:57 30012 ERR 11000 vectorio SYSTEM ERROR on devno 6

2006-02-02 01:08:57 30012 ERR 15 IOMan Access error on DualLog volume 1 blockno 1: Error during seek or write

2006-02-02 01:08:57 30012 ERR 15 IOMan Access error on DualLog volume 1 blockno 1: ÐÃþ@Äþ@VP(3ð-@op Data IO, Pages: 1 IO:

2006-02-02 01:08:57 30012 ERR 17 IOMan EMERGENCY SHUTDOWN

2006-02-02 01:08:57 30012 ERR 8 Admin ERROR '' CAUSED EMERGENCY SHUTDOWN

2006-02-02 01:15:37 --- Starting GMT 2006-02-02 00:15:37 7.5.0 Build 015-121-074-675

2006-02-02 01:15:48 31748 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:22:27 --- Starting GMT 2006-02-02 00:22:27 7.5.0 Build 015-121-074-675

2006-02-02 01:22:38 499 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:29:22 --- Starting GMT 2006-02-02 00:29:22 7.5.0 Build 015-121-074-675

2006-02-02 01:29:32 1443 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:36:14 --- Starting GMT 2006-02-02 00:36:14 7.5.0 Build 015-121-074-675

2006-02-02 01:36:25 3840 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:43:29 --- Starting GMT 2006-02-02 00:43:29 7.5.0 Build 015-121-074-675

2006-02-02 01:43:43 4770 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:50:41 --- Starting GMT 2006-02-02 00:50:41 7.5.0 Build 015-121-074-675

2006-02-02 01:50:56 7128 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 01:57:22 --- Starting GMT 2006-02-02 00:57:22 7.5.0 Build 015-121-074-675

2006-02-02 01:57:36 8049 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 02:04:17 --- Starting GMT 2006-02-02 01:04:17 7.5.0 Build 015-121-074-675

2006-02-02 02:04:33 9795 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 02:11:13 --- Starting GMT 2006-02-02 01:11:13 7.5.0 Build 015-121-074-675

2006-02-02 02:11:29 11303 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 02:27:22 --- Starting GMT 2006-02-02 01:27:22 7.5.0 Build 015-121-074-675

2006-02-02 02:27:34 3336 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 02:44:01 --- Starting GMT 2006-02-02 01:44:01 7.5.0 Build 015-121-074-675

2006-02-02 02:44:13 3359 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 03:01:04 --- Starting GMT 2006-02-02 02:01:04 7.5.0 Build 015-121-074-675

2006-02-02 03:01:16 3338 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 09:13:08 --- Starting GMT 2006-02-02 08:13:08 7.5.0 Build 015-121-074-675

2006-02-02 09:13:22 2854 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 09:17:26 --- Starting GMT 2006-02-02 08:17:26 7.5.0 Build 015-121-074-675

2006-02-02 09:17:37 5459 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 09:45:28 --- Starting GMT 2006-02-02 08:45:28 7.5.0 Build 015-121-074-675

2006-02-02 09:45:36 7076 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 09:56:55 --- Starting GMT 2006-02-02 08:56:55 7.5.0 Build 015-121-074-675

2006-02-02 09:57:06 8252 ERR 28 Log Savepoint iosequence not found on log

2006-02-02 10:03:07 --- Starting GMT 2006-02-02 09:03:07 7.5.0 Build 015-121-074-675

2006-02-02 10:03:18 8811 ERR 28 Log Savepoint iosequence not found on log

estos106:/sapdb/data/wrk/CRP #

[roland_mallmann]

Hi Fredrik,

can you tell me if you've made any full data and log backups?

Are you sure the Logvolume was filled for 100%? Do you see any proof of that in either the knldiag or knldiag.err file?

Can you please logon to your instance with dbmcli and issue the following commands:

1. Setting the state of the db instance to <b>admin</b>

dbmcli -d <SID> -U c db_admin

2. Reading the <b>restartinfo</b> while in state <b>admin</b>

dbmcli -d <SID> -U c db_restartinfo

Kind regards,

Roland

[Former Member]

Hello,

The problem was caused by another system filling up the logfilsystem again to 100%. After fixing the main problem the maxdb was able to recover automatically.

//Fredrik

[roland_mallmann]

Hi Fredrik,

ok, so my first assumption (i.e. not the actual log was filled to 100%, but the filesystem on which it resides) was correct. Good to hear your issue's been resolved.

Anyway, I would recommed you to upgrade to the newest available MaxDB 7.5 version.

Regards,

Roland