Backup halts at BR0307I - Shutting down database instance PRD

[arya_biswas]

Dear Gurus,

For last 20 days our backups have failing 5 times with same error.

Backup devie - LTO, type - Offline

Case :

Backup error : BR0307I Shutting down database instance PRD ...

It halts here and also 'R3trans -d' return code is 0012.

A manual restart of the server done.

Observation - Almost Every-time the next days backup completed successfully. So I think the issue of permission set, space, brtools is not there.

System :

OS : AIX 5.3

DB : Oracle 10.2 (DB is mounted on SAN)

SAP : ECC 6.0

Please help me. Because I need to resolve it to ensure the success of today's Offline backup. Because there is no guarantee that it will be ok toady.

//Arya

[Former Member]

Try to find out why shutdown was hanging. What is in the Oracle alert log file?

And how long did you wait before restarting the server? Time needed for an Oracle shutdown may vary. I have seen anything from a minute to almost an hour.

regards

[Former Member]

Hi,

Please check for the permission thoroughly once. Also please check your R3 trans level also. If possible update the same also and try.

Regards

D.Mukunthan

[arya_biswas]

Hi Mukunthan,

Thanks for the reply.

I have checked the permissions and it's ok. The 'R3trans' is latest as we have upgraded our Kernel set recently.

Can it be an issue with LTO (although it is with very little chance as backup halts at shutting down phase after checking LTO and Tape.)

Also can it be a case with the SAN storage?

//Arya

[Former Member]

Hello Arya,

even if you look at SAP Note : 17163 with BR0307I error, it says The database instance is shut down by BRBACKUP using SQLDBA.

Also please paste the last time failed offline backup log file from /oracle/<SID>/sapbackup directory

Rgds,

Durga.

[arya_biswas]

Hello Arya,

even if you look at SAP Note : 17163 with BR0307I error, it says The database instance is shut down by BRBACKUP using SQLDBA.

Also please paste the last time failed offline backup log file from /oracle/<SID>/sapbackup directory

Hi Durga,

The log :

Detail log: behjoeel.aff

BR0051I BRBACKUP 7.00 (24)

BR0055I Start of database backup: behjoeel.aff 2011-12-06 23.30.03

BR0484I BRBACKUP log file: /oracle/PRD/sapbackup/behjoeel.aff

BR0477I Oracle pfile /oracle/PRD/102_64/dbs/initPRD.ora created from spfile /oracle/PRD/102_64/dbs/spfilePRD.ora

BR0101I Parameters

Name Value

oracle_sid PRD

oracle_home /oracle/PRD/102_64

oracle_profile /oracle/PRD/102_64/dbs/initPRD.ora

sapdata_home /oracle/PRD

sap_profile /oracle/PRD/102_64/dbs/initPRD.sap

backup_mode ALL

backup_type offline_force

backup_dev_type util_file

util_par_file /oracle/PRD/102_64/dbs/initPRD.utl

system_info prdadm/oraprd tegaprd AIX 3 5 000A92FAD700

oracle_info PRD 10.2.0.2.0 8192 31521 3387247473 tegaprd WE8DEC UTF8

sap_info 700 SAPPRD PRD H1215870689 R3_ORA 0020188619

make_info rs6000_64 OCI_102 Apr 21 2007

command_line brbackup -u / -jid ALGOF20111012233000 -c force -t offline_force -m all -p initPRD.sap -v PRDB02 -a -c force -p initPRD.sap -cds

BR0116I ARCHIVE LOG LIST before backup for database instance PRD

Parameter Value

Database log mode Archive Mode

Automatic archival Enabled

Archive destination /oracle/PRD/oraarch/PRDarch

Archive format %t_%s_%r.dbf

Oldest online log sequence 31518

Next log sequence to archive 31521

Current log sequence 31521 SCN: 3387247473

Database block size 8192 Thread: 1

Current system change number 3387249957 ResetId: 550258417

BR0118I Tablespaces and data files

Tablespace TS-Status F-Status File Size Id. Device Link Type MaxSize IncrSize BlkSize

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata3/prd_1/prd.data1 2747277312 2 3735556 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE /oracle/PRD/sapdata3/prd_12/prd.data12 12884910080 28 3735556 NOLINK FILE 0 0 8192

PSAPPRD ONLINE* ONLINE /oracle/PRD/sapdata3/prd_13/prd.data13 12884910080 15 3735556 NOLINK FILE 0 0 8192

PSAPPRD ONLINE* ONLINE /oracle/PRD/sapdata3/prd_14/prd.data14 12884910080 16 3735556 NOLINK FILE 0 0 8192

PSAPPRD ONLINE* ONLINE /oracle/PRD/sapdata3/prd_15/prd.data15 12884910080 17 3735556 NOLINK FILE 0 0 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata3/prd_2/prd.data2 2726305792 3 3735556 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata3/prd_3/prd.data3 2810191872 4 3735556 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata3/prd_4/prd.data4 3019907072 5 3735556 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_10/prd.data10 2196774912 24 3735557 NOLINK FILE 34359721984 8192 8192

PSAPPRD ONLINE* ONLINE /oracle/PRD/sapdata4/prd_11/prd.data11 13958651904 27 3735557 NOLINK FILE 0 0 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_5/prd.data5 2894077952 6 3735557 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_6/prd.data6 2789220352 7 3735557 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_7/prd.data7 2579505152 8 3735557 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_8/prd.data8 2474647552 9 3735557 NOLINK FILE 10485760000 20971520 8192

PSAPPRD ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd_9/prd.data9 2327846912 10 3735557 NOLINK FILE 10485760000 20971520 8192

PSAPPRD700 ONLINE* ONLINE+ /oracle/PRD/sapdata1/prd700_2/prd700.data2 6853500928 11 3735554 NOLINK FILE 34359721984 536870912 8192

PSAPPRD700 ONLINE* ONLINE+ /oracle/PRD/sapdata1/prd700_3/prd700.data3 6853500928 12 3735554 NOLINK FILE 34359721984 536870912 8192

PSAPPRD700 ONLINE* ONLINE+ /oracle/PRD/sapdata1/prd700_4/prd700.data4 6853500928 13 3735554 NOLINK FILE 34359721984 536870912 8192

PSAPPRD700 ONLINE* ONLINE+ /oracle/PRD/sapdata1/prd700_5/prd700.data5 6853500928 14 3735554 NOLINK FILE 34359721984 536870912 8192

PSAPPRD700 ONLINE* ONLINE+ /oracle/PRD/sapdata4/prd700_1/prd700.data1 31476162560 26 3735557 NOLINK FILE 34359721984 8192 8192

PSAPPRDUSR ONLINE* ONLINE+ /oracle/PRD/sapdata1/prdusr_1/prdusr.data1 41951232 22 3735554 NOLINK FILE 10485760000 20971520 8192

PSAPTEMP ONLINE# ONLINE /oracle/PRD/sapdata3/temp_1/temp.data1 2147491840 -1 3735556 NOLINK FILE 0 0 8192

PSAPUNDO ONLINE- ONLINE /oracle/PRD/sapdata2/undo_1/undo.data1 16106135552 25 3735555 NOLINK FILE 0 0 8192

SYSAUX ONLINE* ONLINE+ /oracle/PRD/sapdata1/sysaux_1/sysaux01.dbf 1048584192 29 3735554 NOLINK FILE 2097152000 104857600 8192

SYSTEM ONLINE% SYSTEM /oracle/PRD/sapdata1/system_1/system.data1 2147491840 1 3735554 NOLINK FILE 0 0 8192

BR0119I Redo log files

File Size Group Device Status Link Type

/oracle/PRD/origlogA/log_g11m3.dbf 52429312 1 2949122 INUSE NOLINK FILE

/oracle/PRD/origlogA/log_g11m1.dbf 52429312 1 2949122 INUSE NOLINK FILE

/oracle/PRD/mirrlogA/log_g11m2.dbf 52429312 1 2949124 INUSE NOLINK FILE

/oracle/PRD/origlogB/log_g12m1.dbf 52429312 2 2949123 INUSE NOLINK FILE

/oracle/PRD/origlogA/log_g12m3.dbf 52429312 2 2949122 INUSE NOLINK FILE

/oracle/PRD/mirrlogB/log_g12m2.dbf 52429312 2 2949125 INUSE NOLINK FILE

/oracle/PRD/origlogA/log_g13m3.dbf 52429312 3 2949122 INUSE NOLINK FILE

/oracle/PRD/origlogA/log_g13m1.dbf 52429312 3 2949122 INUSE NOLINK FILE

/oracle/PRD/mirrlogA/log_g13m2.dbf 52429312 3 2949124 INUSE NOLINK FILE

/oracle/PRD/origlogB/log_g14m1.dbf 52429312 4 2949123 INUSE NOLINK FILE

/oracle/PRD/mirrlogB/log_g14m3.dbf 52429312 4 2949125 INUSE NOLINK FILE

/oracle/PRD/mirrlogB/log_g14m2.dbf 52429312 4 2949125 INUSE NOLINK FILE

BR0120I Control files

File Size Id. Device Link Type

/oracle/PRD/origlogA/cntrl/cntrlPRD.dbf 9756672 0 2949122 NOLINK FILE

/oracle/PRD/saparch/cntrl/cntrlPRD.dbf 9756672 0 2949126 NOLINK FILE

/oracle/PRD/sapdata1/cntrl/cntrlPRD.dbf 9756672 0 3735554 NOLINK FILE

BR0280I BRBACKUP time stamp: 2011-12-06 23.30.04

BR0057I Backup of database: PRD

BR0058I BRBACKUP action ID: behjoeel

BR0059I BRBACKUP function ID: aff

BR0110I Backup mode: ALL

BR0077I Database files for backup:

/oracle/PRD/origlogA/log_g11m3.dbf

/oracle/PRD/origlogB/log_g12m1.dbf

/oracle/PRD/origlogA/log_g13m3.dbf

/oracle/PRD/origlogB/log_g14m1.dbf

/oracle/PRD/origlogA/cntrl/cntrlPRD.dbf

BR0061I 30 files found for backup, total size 164666.502 MB

BR0143I Backup type: offline_force

BR0130I Backup device type: util_file

BR0109I Files will be saved by backup utility

BR0289I BRARCHIVE will be started at the end of processing

BR0134I Unattended mode with 'force' active - no operator confirmation allowed

BR0280I BRBACKUP time stamp: 2011-12-06 23.30.04

BR0307I Shutting down database instance PRD ...

BR0280I BRBACKUP time stamp: 2011-12-07 10.34.14

BR0261E BRBACKUP cancelled by signal 13

BR0054I BRBACKUP terminated with errors

I have checked the note already.

Please suggest.

//Arya

[Former Member]

Hello Arya,

As per the attached log says Backup device type: util_file instead value should be as tape i belive (if you are using LTO tape drive). Please check this parameter in your init file once.

Rgds,

Durga.

[arya_biswas]

Hello Durga,

Yeah the device points to the util file. As we have two backups going on everyday.

1. Offline to LTO

2. Online to Tivoli.

So the device points to the util file, it contains the device type. First it matches the tape name mentioned in db13, if nothing mentioned it goes to Tivoli.

I think if it was the reason then it should have happen everyday, but it is occurring in an uneven pattern i.e. 5 times in 20 days.

//Arya

Edited by: Arya Biswas on Dec 13, 2011 8:52 AM