Frequent ORA-00308 in Oracle Data Guard Scenario

[Former Member]

Hi All,

We have oracle dataguard configured for DR for 9 production systems.

ODG works fine for all the systems, however for SAP SCM system I frequently get ORA-00308 in primary system .

The redo log generation rate in this system is high (Its like 70 logs 600MB each per day)

I get a gap of 2 to 3 logs & have to fix it manually by transfering the logs & register them manually.

Error in alert log of primary

ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Tue Mar  8 20:36:21 2011
ORACLE Instance SCP - Archival Error. Archiver continuing.
Tue Mar  8 20:36:21 2011
ARC0: Attempting destination LOG_ARCHIVE_DEST_2 network reconnect (3135)
ARC0: Destination LOG_ARCHIVE_DEST_2 network reconnect abandoned
Tue Mar  8 20:36:21 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc0_25158.trc:
ORA-03135: connection lost contact
FAL[server, ARC0]: FAL archive failed, see trace file.
Tue Mar  8 20:36:21 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc0_25158.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Tue Mar  8 20:36:21 2011
ORACLE Instance SCP - Archival Error. Archiver continuing.
Tue Mar  8 20:36:21 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc3_25164.trc:
ORA-00308: cannot open archived log '/oracle/SCP/oraarch/SCParch1_45166_698614348.dbf'
ORA-27037: unable to obtain file status
HPUX-ia64 Error: 2: No such file or directory
Additional information: 3
Tue Mar  8 20:36:21 2011
FAL[server, ARC3]: FAL archive failed, see trace file.
Tue Mar  8 20:36:21 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc3_25164.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Tue Mar  8 20:36:21 2011
ORACLE Instance SCP - Archival Error. Archiver continuing.
Tue Mar  8 20:36:52 2011
Redo Shipping Client Connected as PUBLIC
-- Connected User is Valid
Tue Mar  8 20:36:52 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc2_25162.trc:
ORA-00308: cannot open archived log '/oracle/SCP/oraarch/SCParch1_45166_698614348.dbf'
ORA-27037: unable to obtain file status
HPUX-ia64 Error: 2: No such file or directory
Additional information: 3
Tue Mar  8 20:36:52 2011
FAL[server, ARC2]: FAL archive failed, see trace file.
Tue Mar  8 20:36:52 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc2_25162.trc:
ORA-16055: FAL request rejected
ARCH: FAL archive failed. Archiver continuing
Tue Mar  8 20:36:52 2011
ORACLE Instance SCP - Archival Error. Archiver continuing.
Tue Mar  8 20:36:52 2011
Errors in file /oracle/SCP/saptrace/background/scp_arc3_25164.trc:
ORA-00308: cannot open archived log '/oracle/SCP/oraarch/SCParch1_45167_698614348.dbf'
ORA-27037: unable to obtain file status
HPUX-ia64 Error: 2: No such file or directory
Additional information: 3

v$archived_log query on DR gives the following output

 SEQUENCE# ARC APP
-
 --- ---
     45155 YES YES
     45156 YES YES
     45157 YES YES
     45158 YES YES
     45159 YES YES
     45160 YES YES
     45161 YES YES
     45162 YES YES
     45163 YES YES
     45164 YES YES
     45165 YES YES
 SEQUENCE# ARC APP
-
 --- ---
     45168 YES NO
     45169 YES NO
     45170 YES NO
     45171 YES NO
     45172 YES NO
     45173 YES NO
     45174 YES NO
     45175 YES NO
     45176 YES NO
     45177 YES NO
     45178 YES NO

Now, how to get to the root cause of this issue?

Is it because of the network?

Also, there is an archive policy which runs on primary , it deletes archives every 8 hours

Thanks,

Salman

[stefan_koehler]

Hello Salman,

what is your transport mechanism or better said how do you send your redo data (LGWR SYNC or ASYNC / ARCn)?

What oracle release do you use (including patch number, etc.)?

There are several known issue like bug #6851669.

Too less information ...

Regards

Stefan

[Former Member]

Hi Stefan,

For this system, we have set up Arcn to ship the logs. For all other systems its LGWR async.

Oracle version is :

Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 - 64bit

PL/SQL Release 10.2.0.4.0 - Production

CORE 10.2.0.4.0 Production

TNS for HPUX: Version 10.2.0.4.0 - Production

NLSRTL Version 10.2.0.4.0 - Production

Please let me know if you need some more information

Thanks,

Salman

[stefan_koehler]

Hello Salman,

ok thanks. It seems like the SQL*net packet is altered by anyone (like firewall, etc.).

Metalinknote #739522.1 describes howto check and trace this. You need to do further investigations on that area.

Regards

Stefan