SAP Maxdb patch upgrade problems starting

[Former Member]

Hi SAP,

Yesterday, I started patching our Solution Manager system SID with maxdb server patch from 7.8.01.14 to  7.8.02.35 however at the end of the upgrade, I ran into a problem.

I have added 3 data files before the upgrade. They got added fine as I can see from dbm.prt file however i got the error below at the end of the upgrade where its trying to detach the newly created data files. Now maxdb wont start on its own at the end of the upgrade and doesn't finish loading system tables and catalog check which happens at the end of the server upgrade. I can start db manually and checked R3trans -d connectivity it works fine. However when i start the SDBUPD command to finish the server patch upgrade, i am unable to get it to work due to the detach failing.

Do I need to delete the newly created data files ??. We have SPS upgrade planned and we are stuck with the problem. Your quick help

would be appreciated.

Error Message

------------------------------------------------------------------------------------------------------------------------------------------------

Checking upgrade...

Running finalize check...

Current database state is OFFLINE

Checking parameters...

Switching database state to ADMIN

Switching database state to ONLINE

ERR:  Upgrade failed

ERR:    error upgrading

ERR:      Finalize upgrade failed: Current database state is OFFLINE

Checking parameters...

Switching database state to ADMIN

Switching database state to ONLINE

Cannot switch database mode, error during dbm command:

db_online

ERR

-24988,ERR_SQL: SQL error

-902,I/O error

3,Database state: OFFLINE

Internal errorcode, Error code 9050 "disk_not_accessible"

20017,RestartFilesystem failed with 'I/O error'

Looking for error messages in knldiag

------------------------------------------------------------------------------------------------------------------------------------------------

The volumes I added just before starting patching are sapdb/SID/sapdata/DISKD0017,/sapdb/SID/sapdata/DISKD0018,/sapdb/SID/sapdata/DISKD0019.  I can see the new data files via DBM and also dbmcli command line so there is no discrepancy with respect to the data files that got added. I dont think it got registered correctly. When I check dbm.prt it shows return code zero when i actually executed the volume additions however i see error below repeatedly in the dbm.prt file

2013-07-29 15:17:58      12669 INF        421 DBMSrv   Command 'param_getvolume ...' was executed since 2013-07-29 15:17:58.

2013-07-29 15:17:58      12669 ERR     -24580 DBMSrv   ERR_COMMAND_FAILED: Command 'param_getvolume' has ended and failed with return code -24979.

                         12669 ERR     -24999 DBMSrv   ERR_ERR: Common error

Already logged a call but thought try my luck here as well.

Regards

Kalyan

[Former Member]

Seems to me that the config file under /sapdb/SID/sapdata/config is corrupt.  I can see the errors below looping even before I added some data files yesterday

2013-07-29 15:17:58      12669 ERR     -24580 DBMSrv   ERR_COMMAND_FAILED: Command 'param_getvolume' has ended and failed with return code -24979.

                         12669 ERR     -24999 DBMSrv   ERR_ERR: Common error

...i can see errors from 29th July at 15:00 hours and I added the data file at 19:46 yesterday so i need to restore a config file version thats before 15:00 hours time stamp. I can see one file which is a copy made a week ago. I did not make any config changes in the last couple of weeks so i can restore this file.

Now even if i restore the config file from before, it wont have the newly created data files. I have to add them manually. Any thoughts how i can restore old config file and add the newly created volumes manually in a consistent manner ?  Has any one done this before. Is this a recommended approach given my situation.

Regards

Kalyan

[Former Member]

Hi.

Perhaps check this doc pg 96

http://www.sapdb.org/7.4/pdf/dbmcli_74eng.pdf

Regards,

Johan

[Former Member]

Thanks. I am reviewing the commands especially *directput ones.

Now i am wondering how i can validate my conclusion that the OS parameter sid is corrupt under /sapdb/SID/sapdata/config.

Does the command param_getvolsall DATA read from this file and if so, i can see all the data files correctly setup. If not is there any other way I can validate my theory on the parameter file being corrupt ?. Any thoughts here please.

Regards

Kalyan

[Former Member]

Hi,

Check the following links:

http://www.sdn.sap.com/irj/scn/downloads?rid=/library/uuid/809ed04c-b5d3-2b10-5393-fd8cb5e71ad7

http://scn.sap.com/docs/DOC-7880?rid=/webcontent/uuid/801f1931-bbd9-2b10-e4be-8c2ae9afe7bf

There is tool in first link it seems to verify the file.

Regards,

Johan

[Former Member]

thanks.

I have already checked this. The db has to be online for this tool to work. Unfortunately i cant start the db consistantly

Note 1111426 - Parameter check for SAP MaxDB/liveCache instances

Regards

Kalyan

[yashwanthkumar_kethineed]

Hello Kalyan,

Please post the output of DBM command  param_getvolsall

Do you have complete data backup taken before the upgrade ?

What is the output of "sdbregview -l"

What is the output of "xinstinfo <SID>"

Check the database packages for consistency - Run as root/administrator - sdbverify and see if there are any inconsistent packages.

[Former Member]

Hi Yashwanth,

param_getvolsall outputs all volumes including the ones i added just before the problems started.

sdbregview -l shows all components upgraded. 

xinstinfo sid did not report any problems either.

The problem as i see is at the end of the server patch are 3 pending steps

1) attach/detach all volumes)

2) load system tables

3) verify catalog.

steps 2 and 3 will be successfull if step 1 were to run fine.

step1 fails at detaching volumes 17,18 and 19 which are the newly created volumes.  The volumes perse are created as dbm.prt output shows return code zero when executing volume creation steps. The volumes are visible in complete from OS level and from DBM. Somehow this is not reflecting in the config file i am guessing due to corruption and other reasons and that may be the reason for not being able to detach new volumes or so I think.

I can restore previous config file and can run param_directput to manually update config file to reflect the volumes correctly.  I just need sap/scn team to validate this so I can try this and have support from sap just incase things go south

As to the backups, i have 2 good backups. I am worried if the problem occurs again, i want to be in control so restore would be the last resort for me.

Regards

Kalyan

[yashwanthkumar_kethineed]

Hello Kalyan,

-> Are you SAP customer ?

-> Did you run sdbverify as root/admin to see if there are any inconsistencies in the software ?

-> It is not possible to validate the workaround suggested by you without checking the logs - If you have created a CSN ticket for this issue already, let me know - I will check the log files.

Regards,Yashwanth