Oracle Startup error - unable to open file

[Former Member]

Hi ,

Last week we had a preventive maintenance shutdown at our data center. After that activity when we tried to bring back our servers we started facing some issue in our Sandbox system which is running on Linux & Oracle.

We couldn't start the oracle server , we tried through sqlplus,

SQL> startup;

ORACLE instance started.

Total System Global Area 2298478592 bytes

Fixed Size                  2085616 bytes

Variable Size            1157631248 bytes

Database Buffers         1124073472 bytes

Redo Buffers               14688256 bytes

ORA-00205: error in identifying control file, check alert log for more info

The alert log file has error "Permission denied"

Thu Jan  2 17:35:02 2014

ORA-00202: control file: '/oracle/R01/origlogB/cntrl/cntrlR01.dbf'

ORA-27041: unable to open file

Linux-x86_64 Error: 13: Permission denied

Additional information: 2

Thu Jan  2 17:35:02 2014

ORA-205 signalled during: ALTER DATABASE   MOUNT...

But cntrlR01.dbf has file permission 755...

ls -ltr

total 15044

-rwxr-xr-x 1 orar01 dba 15384576 Jan  2 16:44 cntrlR01.dbf

Even we tried after restoring the DB , but still facing the same issue. Any idea , how we can resolve this?

[Former Member · Accepted solution]

Finally both instance are now up!

The issue is with OS. So Installed OS again , did SAP installation and then restored the DB.

[Former Member]

Hi,

Please provide your alert log file to find the solution.

Thank you

Mahendran

[Reagan]

Hello

Are you able to access the control files under orar01 user ?

su - orar01

ls -la /oracle/R01/origlogA/cntrl/cntrlR01.dbf

ls -la /oracle/R01/origlogB/cntrl/cntrlR01.dbf

ls -la /oracle/R01/sapdata1/cntrl/cntrlR01.dbf

If possible stop all applications on the server unmount and mount the file systems back.

Do a file system check.

If everything is OK then check the permissions.

Regards

RB

[Former Member]

SAPSERVER:orar01 130> ls -la /oracle/R01/origlogA/cntrl/cntrlR01.dbf

-rwxr-xr-x 1 orar01 dba 15384576 Jan  3 17:48 /oracle/R01/origlogA/cntrl/cntrlR01.dbf

SAPSERVER:orar01 131> ls -la /oracle/R01/origlogB/cntrl/cntrlR01.dbf

-rwxr-xr-x 1 orar01 dba 15384576 Jan  3 17:49 /oracle/R01/origlogB/cntrl/cntrlR01.dbf

SAPSERVER:orar01 132> ls -la /oracle/R01/sapdata1/cntrl/cntrlR01.dbf

-rwxr-xr-x 1 oran01 dba 15384576 Jan  3 17:46 /oracle/R01/sapdata1/cntrl/cntrlR01.dbf

we tried mounting and unmounting filesystems already.

[Former Member]

Please provide output of mount command.

[Former Member]

Hi,

as both instances are affected, could you please list here the ownership/permisson of the /oracle and the /oracle/<SID> filesystems as well?

Thanks

Tamás

[Reagan]

Hello

Is the control file /oracle/R01/origlogB/cntrl/cntrlR01.dbf present at the OS level ?

ls -la /oracle/R01/origlogB/cntrl/cntrlR01.dbf

If yes the check the permission and ownership of the file and/or the directory or file system.

Regards

RB

[Former Member]

Hi,

Give 755 permissions to the parent director and all contents (not only to control file).

Then try to start database and check.

If same issue, then just change the control file location to some other place copy control file to new location and then try to start database.

Regards,

Nick Loy

[Former Member]

Hi vignesh,

1.What it is that maintenace task you did explain?

2.Have you deleted any user and recreated?

3.Have you deleted any files or folders?

From your log oracle is not able to write alertlog also, see the error

The alert log file has error "Permission denied" - check the permission

regards,

Suresh Daniel.

[former_member188883]

Hi Vignesh,

Post restoration did you recreate the control files ?

If not recreate the same and then startup database.

Regards,

Deepak Kori

[Former Member]

Hi,

Can you please check the below note.

1894803 - Oracle database instance does not start

If it doesn't help can you please post the alert log.

Thanks

RishI Abrol

[Former Member]

After your database is in no mount state ,

give this command :

show parameter control_files... it will give you location of the control files for your DB .

After that check the permissions of the files, ideally in your situation it should work.

To remove any OS level issues , give 777 to all the control files and then try

SQL> alter database mount

Check the alert log and let us know the output .

Thanks

Rishi

[Former Member]

Hi Rishi,

SQL> show parameter control_files;

NAME                                 TYPE

------------------------------------ ---------------------------------

VALUE

------------------------------

control_files                        string

/oracle/R01/origlogA/cntrl/cnt

rlR01.dbf, /oracle/R01/origlog

B/cntrl/cntrlR01.dbf, /oracle/

R01/sapdata1/cntrl/cntrlR01.db

f

We changed the the files permission from 755 to 777 , and

SQL> alter database mount;

alter database mount

*

ERROR at line 1:

ORA-00205: error in identifying control file, check alert log for more info

Alert log still has the same error :

Fri Jan  3 11:38:02 2014

alter database mount

Fri Jan  3 11:38:02 2014

ORA-00202: control file: '/oracle/R01/origlogA/cntrl/cntrlR01.dbf'

ORA-27041: unable to open file

Linux-x86_64 Error: 13: Permission denied

Additional information: 2

Fri Jan  3 11:38:05 2014

ORA-205 signalled during: alter database mount...

[Former Member]

Hi,

Can you please check that the size of all the control file are same in all the three location.

Hope that the control file at this location is not corrupt.

ORA-00202: control file: '/oracle/R01/origlogA/cntrl/cntrlR01.dbf'

Thanks

Rishi Abrol

[former_member188883]

Hi Vignesh,

What is ur SAP system ID on which issue is faced ?

Secondly check the path for control files as described in initSID.ora file.

Regards,

Deepak Kori

[Former Member]

Hi Vignesh

Thanks for the update.

Seems that there is an issue with this control file : '/oracle/R01/origlogA/cntrl/cntrlR01.dbf' .

We need to try to bring the DB up using the other 2 control files.

Do this :

a) Show parameter pfile

Check if you are using the spfile or pfile.

b) modify your pfile  to remove the control_files entry for this file.

Basically control_files should point to only this location as below. take backup of your current pfile.

control_files='/oracle/R01/origlogB/cntrl/cntrlR01.dbf', '/oracle/R01/sapdata1/cntrl/cntrlR01.dbf'

c) After this -

SQL> create spfile from pfile ;

after that issue - startup  command and then we will check the out

d)  If this opens - our assumption is fine.

Thanks

Rishi

[Former Member]

Control file size is same in all the locations

-rwxrwxrwx 1 orar01 dba 15384576 Jan  2 16:44 cntrlR01.dbf

-rwxrwxrwx 1 orar01 dba 15384576 Jan  2 16:44 cntrlR01.dbf

-rwxrwxrwx 1 orar01 dba 15384576 Jan  2 16:44 cntrlR01.dbf

[Former Member]

We have 2 SID's in that box, facing issue in both instances.

[Former Member]

Hi,

did you check the sap not that was posted earlier.

can you please also explain more when issue started.

Thanks

Rishi Abrol

[Former Member]

Hi Vignesh

Thanks for the quick responses.

Please try to start up the DB using the other 2 control files so that we can isolate out the issue..

Thanks

Rishi

[Former Member]

Dear Rishi,

I followed your steps,

SQL> show parameter pfile

NAME                                 TYPE

------------------------------------ ---------------------------------

VALUE

------------------------------

spfile                               string

/oracle/R01/102_64/dbs/spfileR

01.ora

used vi editor to modify spfileR01.ora to remove '/oracle/R01/origlogA/cntrl/cntrlR01.dbf' and saved.

but,

SQL> create spfile from pfile;

It creates entry '/oracle/R01/origlogA/cntrl/cntrlR01.dbf' again in spfileR01.ora

I tried even running without above query , but in both cases I get the same old error.

[Former Member]

Hi,

You cannot edit spfile.

First create pfile from spfile and then edit pfile >> Then start your database using pfile.

sqlplus>> create pfile from spfile;

Edit pfile (Change the controlfile path to some other location

Copy controlfile to new location

sqlplus >> startup pfile=path of pfile

Paste the errors, if any.

Regards,

Nick Loy

[Former Member]

We had a preventive annual power shutdown at Data center , we did nothing expect stopping SAP , Oracle instance.

[Former Member]

Hi Vignesh

As mentioned by Nick , you should never edit the SPFILE... its a dynamic file.. always edit the PFILE..

Please let us know how you go !

Thanks

Rishi

[Former Member]

Hi,

As the database was not stopped correctly hope that its not corrupt.

Can you please try the below command.

startup mount

recover database;

alter database open;

Thanks

Rishi Abrol

[Former Member]

Hi Rishi,

I don't think it will allow to mount database if you have an issue with control file.

Vignesh - Just follow the steps mentioned in my last post, edit pfile (not spfile) and copy controlfile to new location and then try to start your database using pfile (startup pfile=path).

Regards,

Nick Loy

[Former Member]

Hi Nick

Yes you are right , if the control files mentioned in the parameter files is not correct , then DB will not mount ...

Hence wanted to try to mount the DB using the control files located in the other locations... if the DB mounts , then it is issue with the specific control file.. So wanted to isolate the error ...

if the DB fails to mount , then I think he might need to go to the Linux Team as the issue started in the 1st place after the data center outage...

Thanks

Rishi

[Former Member]

Yes Rishi,

But multiple hit and tries unnecessarily creates lot of confusion to the executor.

If alert log is showing an error with permissions, then we should focus on eliminating that.

If replacing the controlfile to other location helps, then the issue will get resolved. If it doesn't help then next action can be taken.

Just to avoid confusion to the executor.

Regards,

Nick Loy

[Former Member]

Hi Nick

Cannot agree more with you !

Waiting for the results from the executor !!

Thanks

Rishi

[Former Member]

Seems all the control files are courrpted. Is there is way to create control file with while we running DB in nomount option.

Or restore it from backup file cntrlR01.dbf.Z ??  using brtools ?

[Former Member]

Seems all the control files are courrpted

What is the latest error after replacing the controlfile in new location?

Regards,

Nick Loy

[Former Member]

Guys,

Non of the three control files are working !!! Don't know what damaged them...Now is there is a way to create new control file when database is not mounted! or is it possible to restore control file from backup using brtools (If yes , what is the brtools command for this).

[Former Member]

Hi Nick,

Accept it that the database cant be mounted if the control file are corrupt .

Thanks

Rishi Abrol

[Former Member]

My point was - If issue is with one particular control file/location then it gets resolved by replacing the control file to new location.

If system throws same error even after the control file movement, then there should be some other reason (with corrupted control file etc.).

Regards,

Nick Loy

[Former Member]

Accept it that the database cant be mounted if the control file are corrupt .

Yes, you are right....In fact I never suggested to mount the database here

I was just trying to eliminate the alert log error by moving the control file to some other location.

Regards,

Nick Loy

[Former Member]

Same error

ORA-00202: control file: '/oracle/R01/origlogB/cntrl/cntrlR01.dbf'

ORA-27041: unable to open file

Linux-x86_64 Error: 13: Permission denied

[Former Member]

Hi Vignesh

As the entire issue started after the Linux Data Center Outage , I think you should contact the Linux Team with the permission error.

Also you can try to restore the control file using the command

brrestore -m 0

Thanks

Rishi

[Former Member]

Did you follow the steps mentioned in my post?

It seems still system is looking for control file in same location ???

If you move the control file to some other place it should have highlighted the new location!

Regards,

Nick Loy

[Reagan]

What guarantee that the datafiles are not having the same problem ?

Try to unmount and mount the file systems like I suggested below.

There is no point in doing the same step again and again.

If none of the suggested steps work then start the restore.

Restoring the control files from the backup alone won't work.

Regards

RB

[Former Member]

Hi Nick,

Earlier it was refering to '/oracle/R01/origlogA/cntrl/cntrlR01.dbf' now it is referring '/oracle/R01/origlogB/cntrl/cntrlR01.dbf'...

[Former Member]

Earlier it was refering to '/oracle/R01/origlogA/cntrl/cntrlR01.dbf' now it is referring '/oracle/R01/origlogB/cntrl/cntrlR01.dbf'...

I suggest to go ahead and replace other 2 control files as well.

Regards,

Nick Loy

[Former Member]

Dear Reagan,

We tried after moving the file system to other mount point, in that case also we faced same issue.

My guess was either the control file has corrupted or some issue in OS.

[Former Member]

If issue is with particular mount points where your control file exits, then replacing the control file will help. Try to replace other 2 control files and then check, if issue still persists with data files then there must be some other issue (related to OS/permissions etc.) not with database.

Regards,

Nick Loy

[Former Member]

I dont think any issues at OS level.

May be you have not stopped the database during the maintenance or database not completely down before shutdown.

As suggested by Reagan please restore the database and recover.

[former_member227600]

Could you please rename control file from origlogA & origlog B & than copy control file from sapadata1 & try to start DB .

[Former Member]

I would look at backup restore as final option (worst case scenario), I go with that if system throws same error even after replacing the control files!

All the best!

Regards,

Nick Loy

[Former Member]

I tried all the control files nick.

Fri Jan  3 17:41:44 2014

ORA-00202: control file: '/home/oracntrl/cntrlR01.dbf'

ORA-27041: unable to open file

Linux-x86_64 Error: 13: Permission denied

[Reagan]

Mount the database with only one control file - /oracle/R01/sapdata1/cntrl/cntrlR01.dbf

Create a pfile from spfile and then modify the parameter control_files parameter in the pfile by removing

/oracle/R01/origlogA/cntrl/cntrlR01.dbf and /oracle/R01/origlogB/cntrl/cntrlR01.dbf.

Once done start the database in mount with the pfile.

Regards

RB

[Former Member]

Edited pfile and changed control file location to non oracle mount point.

[Former Member]

Some of your reply mentioned the control file is under /oracle but the latest reply showing /home

You need to first findout where are the control files located from the PFILE.

Make sure you have correct permission and owner for all the control files.

[Former Member]

Hi Vignesh

Let us know the output of the activity and the current status.

Thanks

Rishi

[Former Member]

Hi Guys,

we have tried to bring up Oracle DB with single Control file , after editting initR01.ora

first with

/oracle/R01/origlogA/cntrl/cntrlR01.dbf

then with

/oracle/R01/origlogB/cntrl/cntrlR01.dbf

later

/oracle/R01/sapdata1/cntrl/cntrlR01.dbf

nothing works , then we copied control file to /home/oracntrl ,we tried to bring up the server after editing control file details in initR01.ora. That also failed.

So , We did DB restore and control file restore from backup. Even after that we are getting same issue.

Checked user and group - it is fine. We checked the file system , it is also fine.

Now , I guess the issue is not with the oracle , it must be with OS. So I am sitting along with my Linux team to get it resolved.

[Former Member]

Guys to make sure the issue is not with Oracle , I have asked my Linux team to move Oracle mount point to some other Linux server.

After moving the mount points ,

export ORACLE_HOME=/oracle/R01/102_64/

export ORACLE_SID=R01

cd $ORACLE_HOME

cd bin/

sqlplus / as sysdba

SQL> startup mount;

ORACLE instance started.

Total System Global Area 2298478592 bytes

Fixed Size                  2085616 bytes

Variable Size            1157631248 bytes

Database Buffers         1124073472 bytes

Redo Buffers               14688256 bytes

Database mounted.

So now it is clear the issue is not with Oracle. Will keep you updated.

[Former Member]

Hi Vignesh

Thanks for the information above.

In your last output , it shows as DB as mounted , so the control files issue should be resolved as DB will not mount without using consistent control files.

Did you try a alter database open to check if the DB would open . 

If it opens , then do a show parameter control_files to check the location where the control files are being read fine .

Thanks

Rishi

[Former Member]

hm! strange... request your Linux Admin team to investigate whats causing the issue on original Linux server.. are the both linux system are on same version/patch?

looks like some issue while mounting the FS..

[Former Member]

Dear Rishi,

SQL> show parameter control_files

NAME                                 TYPE        VALUE

------------------------------------ ----------- ------------------------------

control_files                        string      /oracle/R01/origlogA/cntrl/cnt

                                                 rlR01.dbf, /oracle/R01/origlog

                                                 B/cntrl/cntrlR01.dbf, /oracle/

                                                 R01/sapdata1/cntrl/cntrlR01.db

                                                 f

Seems none of the control files has the issue. The issue is with file system or OS.

[Former Member]

Okay - can you now open the DB and then your issue should be fixed.

The original issue reported was definitely due to Linux FS mount

Thanks

Rishi

[Former Member]

Hi,

I suggest you to close the thread if your issue is resolved.

Regards,

Nick Loy