Oracle startup mount or nomount hand with no response

[steven_foo]

We just tried to refresh from our production to test instance on a ecc6/oracle/hp-ux.

We have adapter all the oracle config files and etc.

While starting the below process we notice that startup hang there for more than 1 hour and no response.

There was also no error message. No error found in the alert_SID.log file.

ecc6mcq:oramcq 6> sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.2.0 Production on Tue Jul 28 19:32:06 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup mount

ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance

We tried to start with pfile and the result is the same.

ecc6mcq:oramcq 45> sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.2.0 Production on Tue Jul 28 22:56:54 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup nomount pfile='/oracle/PRD/11202/dbs/initPRD.ora'

ORA-32006: REMOTE_OS_AUTHENT initialization parameter has been deprecated

Any idea what is happening?

[Former Member]

Remore this parameter  REMOTE_OS_AUTHENT and try again.

[Former Member]

Do you have existing processes running on your instance that were not cleanly killed.

Do a 'ps -ef | grep ora' and post the results.

Your path to the pfile shows PRD, but you said this was a QAS (mcq) refresh. This doesn't look right.

[steven_foo]

ecc6mcq:oramcq 14> ps -ef | grep ora

    root  1769  1757  0 17:24:35 ?         0:00 /opt/wbem/lbin/cimprovagt 0 5 10 root HPUXStorageIndicationProviderModule

  oramcq 26880 25188  0 01:15:16 pts/0     0:00 grep ora

  oramcq 20506     1 189 22:56:54 ?        137:37 oraclePRD (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq)))

  oramcq 25188 25167  0 00:40:03 pts/0     0:00 -csh

  oramcq 26879 25188  1 01:15:16 pts/0     0:00 ps -ef

  oramcq 20504 18890  0 22:56:54 pts/1     0:00 sqlplus /as sysdba

  oramcq 18890 11605  0 22:27:23 pts/1     0:00 -csh

ecc6mcq:oramcq 15> ipcs | grep ora

m   26640388 0x00000000 --rw-rw----    oramcq       dba

m     884745 0x00000000 --rw-rw----    oramcq       dba

m     688147 0x00000000 --rw-rw----    oramcq       dba

m     753684 0x00000000 --rw-rw----    oramcq       dba

m     884757 0x00000000 --rw-rw----    oramcq       dba

m     884758 0x00000000 --rw-rw----    oramcq       dba

m    1015831 0x00000000 --rw-rw----    oramcq       dba

m    1146904 0x00000000 --rw-rw----    oramcq       dba

m    1409049 0x00000000 --rw-rw----    oramcq       dba

we start the pfile file for PRD first to recover it and then only we rename it to MCQ.

[steven_foo]

whether we use pfile file or spfile there issue is the same.

startup mount or nomount still hang there and no response at all.

in the alert_SID.log file, we only see the line below. there is nothing after that.

Tue Jul 28 22:57:24 2015

Starting ORACLE instance (normal)

normally oracle will throw out any error if there is an issue. but this one it just stop and hang.

[Former Member]

Those ora process' from your ipcs may be taking all your memory and not allowing your oracle startup.

Check the HPUX equivalent of topas. An ipcs cleanup or reboot could be appropriate.

I would post the last few lines of your alertmcq.log from /oracle/mcq/saptrace/diag/rdbms/mcq/trace/. Do a 'tail -50 alertmcq.log'.

[steven_foo]

1) I have kill all the process

2) clean all ipcs related to oracle

3) reboot the server

4) clear the aler_SID.log file

5) execute the following once server rebooted

ecc6mcq:oramcq 48> setenv ORACLE_SID PRD

ecc6mcq:oramcq 49> env | grep ORA

dbms_type=ORA

ORACLE_SID=PRD

ORACLE_BASE=/oracle

ORACLE_HOME=/oracle/MCQ/112_64

ecc6mcq:oramcq 50> sqlplus "/as sysdba"

SQL*Plus: Release 11.2.0.2.0 Production on Wed Jul 29 02:06:31 2015

Copyright (c) 1982, 2010, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup mount

ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance

6) Output from alert_SID.log

ecc6mcq:oramcq 7> tail -50 alert_PRD.log

Wed Jul 29 02:07:04 2015

Starting ORACLE instance (normal)

7) It still hang and there is no response.Even if I press enter many times on the screen, there is nothing happening.

[steven_foo]

Below is the top process capture for the system.

System: ecc6mcq                                       Wed Jul 29 02:18:44 2015

Load averages: 0.16, 0.18, 0.13

238 processes: 153 sleeping, 85 running

Cpu states:

CPU   LOAD   USER   NICE    SYS   IDLE  BLOCK  SWAIT   INTR   SSYS

0    0.03   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%

1    0.14   8.0%   0.0%  92.0%   0.0%   0.0%   0.0%   0.0%   0.0%

2    0.24   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%

3    0.23   0.0%   0.0%   0.0% 100.0%   0.0%   0.0%   0.0%   0.0%

---   ----  -----  -----  -----  -----  -----  -----  -----  -----

avg   0.16   2.0%   0.0%  23.0%  75.0%   0.0%   0.0%   0.0%   0.0%

System Page Size: 4Kbytes

Memory: 1115968K (817192K) real, 33733636K (33190288K) virtual, 12911848K free  Page# 1/10

CPU TTY  PID USERNAME PRI NI   SIZE    RES STATE    TIME %WCPU  %CPU COMMAND

1   ?  5593 oramcq   213 20 32423M 12800K run      0:48 100.14 91.92 oraclePRD

[Former Member]

It is very unusual that you should be starting a PRD instance on a QAS host. Has this process worked before?

[stefan_koehler]

Hi Steven,

unfortunately the output is a little bit unreadable, but if i see it correctly your Oracle process is running fully on CPU and the overall CPU summary is showing a lot of SYS activity.

However the best way would be to truss / tusc your Oracle process based on your described Oracle behavior (and cmd output). You may need to truss / tusc the whole startup procedure and not only the currently running process to get an idea of the whole system call flow. There are other debug possibilities as well, but i would start with truss / tusc.

Regards

Stefan