on 04-07-2014 7:02 AM
Hi Folks, on the unix shell in a rush I gave the bloody command
find . -type f -mtime +15 -exec rm -f {} \;
I Obviously deleted some files, now that I have done that.
Environment: UNIX, Oracle 10.2, PI server (ABAP+JAVA) stack.
I have backup from 2 months back, I need help in what to do now.
Any help is appreciated
Thanks
HI Folks, as requested I am attaching the last 300 lines of the alert_SID.log file for you to see:
---------------------------------------
Output of the alert_<SID>.log File
SMON started with pid=8, OS id=17649
RECO started with pid=9, OS id=17651
CJQ0 started with pid=10, OS id=17653
MMON started with pid=11, OS id=17655
MMNL started with pid=12, OS id=17657
Wed Apr 09 17:53:53 IST 2014
ALTER DATABASE MOUNT
Wed Apr 09 17:53:57 IST 2014
Setting recovery target incarnation to 1
Wed Apr 09 17:53:57 IST 2014
Successful mount of redo thread 1, with mount id 76606833
Wed Apr 09 17:53:57 IST 2014
Database mounted in Exclusive Mode
Completed: ALTER DATABASE MOUNT
Wed Apr 09 17:53:57 IST 2014
ALTER DATABASE OPEN
Wed Apr 09 17:53:57 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_dbw0_17643.trc:
ORA-01157: cannot identify/lock data file 20 - see DBWR trace file
ORA-01110: data file 20: '/oracle/<SID>/sapdata2/sr3711_7/sr3711.data7'
ORA-27046: file size is not a multiple of logical block size
Additional information: 1
ORA-1157 signalled during: ALTER DATABASE OPEN...
Wed Apr 09 18:02:57 IST 2014
Shutting down instance (immediate)
Wed Apr 09 18:02:57 IST 2014
Shutting down instance: further logons disabled
Wed Apr 09 18:02:57 IST 2014
Stopping background process MMNL
Wed Apr 09 18:02:57 IST 2014
Stopping background process CJQ0
Wed Apr 09 18:02:58 IST 2014
Stopping background process MMON
License high water mark = 1
Wed Apr 09 18:02:59 IST 2014
Job queue slave processes stopped
Wed Apr 09 18:02:59 IST 2014
ALTER DATABASE CLOSE NORMAL
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
Wed Apr 09 18:02:59 IST 2014
ALTER DATABASE DISMOUNT
Completed: ALTER DATABASE DISMOUNT
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Wed Apr 09 18:03:01 IST 2014
Starting ORACLE instance (normal)
Wed Apr 09 18:03:01 IST 2014
Specified value of sga_max_size is too small, bumping to 1795162112
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.5.0.
System parameters with non-default values:
processes = 80
sessions = 160
event = 10191 trace name context forever, level 1
sga_max_size = 1795162112
shared_pool_size = 872415232
shared_pool_reserved_size= 86888153
filesystemio_options = setall
control_files = /oracle/<SID>/origlogA/cntrl/cntrl<SID>.dbf, /oracle/<SID>/origlogB/cntrl/cntrl<SID>.dbf, /oracle/<SID>/sapdata1/cntrl/cntrl<SID>.dbf
control_file_record_keep_time= 30
db_block_size = 8192
db_cache_size = 872415232
compatible = 10.2.0
log_archive_dest = /oracle/<SID>/oraarch/<SID>arch
log_buffer = 14477312
log_checkpoints_to_alert = TRUE
dml_locks = 4000
replication_dependency_tracking= FALSE
undo_management = AUTO
undo_tablespace = PSAPUNDO
_in_memory_undo = FALSE
undo_retention = 43200
recyclebin = off
remote_os_authent = TRUE
remote_login_passwordfile= EXCLUSIVE
job_queue_processes = 1
parallel_execution_message_size= 16384
_table_lookup_prefetch_size= 0
background_dump_dest = /oracle/<SID>/saptrace/background
user_dump_dest = /oracle/<SID>/saptrace/usertrace
core_dump_dest = /oracle/<SID>/saptrace/background
db_name = <SID>
open_cursors = 2000
_sort_elimination_cost_ratio= 10
_b_tree_bitmap_plans = FALSE
star_transformation_enabled= true
query_rewrite_enabled = false
_index_join_enabled = FALSE
_optim_peek_user_binds = FALSE
pga_aggregate_target = 1158508707
_optimizer_mjc_enabled = FALSE
PMON started with pid=2, OS id=18357
PSP0 started with pid=3, OS id=18359
MMAN started with pid=4, OS id=18361
DBW0 started with pid=5, OS id=18363
LGWR started with pid=6, OS id=18365
CKPT started with pid=7, OS id=18367
SMON started with pid=8, OS id=18369
RECO started with pid=9, OS id=18371
CJQ0 started with pid=10, OS id=18373
MMON started with pid=11, OS id=18375
MMNL started with pid=12, OS id=18377
Wed Apr 09 18:03:02 IST 2014
ALTER DATABASE MOUNT
Wed Apr 09 18:03:06 IST 2014
Setting recovery target incarnation to 1
Wed Apr 09 18:03:06 IST 2014
Successful mount of redo thread 1, with mount id 76548758
Wed Apr 09 18:03:06 IST 2014
Database mounted in Exclusive Mode
Completed: ALTER DATABASE MOUNT
Wed Apr 09 18:03:06 IST 2014
ALTER DATABASE OPEN
Wed Apr 09 18:03:06 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_dbw0_18363.trc:
ORA-01157: cannot identify/lock data file 20 - see DBWR trace file
ORA-01110: data file 20: '/oracle/<SID>/sapdata2/sr3711_7/sr3711.data7'
ORA-27046: file size is not a multiple of logical block size
Additional information: 1
ORA-1157 signalled during: ALTER DATABASE OPEN...
Wed Apr 09 18:04:30 IST 2014
Shutting down instance (immediate)
Wed Apr 09 18:04:30 IST 2014
Shutting down instance: further logons disabled
Wed Apr 09 18:04:30 IST 2014
Stopping background process MMNL
Wed Apr 09 18:04:30 IST 2014
Stopping background process CJQ0
Wed Apr 09 18:04:31 IST 2014
Stopping background process MMON
License high water mark = 1
Wed Apr 09 18:04:33 IST 2014
Job queue slave processes stopped
Wed Apr 09 18:04:33 IST 2014
ALTER DATABASE CLOSE NORMAL
ORA-1109 signalled during: ALTER DATABASE CLOSE NORMAL...
Wed Apr 09 18:04:33 IST 2014
ALTER DATABASE DISMOUNT
Completed: ALTER DATABASE DISMOUNT
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
ARCH: Archival disabled due to shutdown: 1089
Shutting down archive processes
Archiving is disabled
Archive process shutdown avoided: 0 active
Wed Apr 09 18:04:40 IST 2014
Starting ORACLE instance (normal)
Wed Apr 09 18:04:40 IST 2014
Specified value of sga_max_size is too small, bumping to 1795162112
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Picked latch-free SCN scheme 3
LICENSE_MAX_USERS = 0
SYS auditing is disabled
ksdpec: called for event 13740 prior to event group initialization
Starting up ORACLE RDBMS Version: 10.2.0.5.0.
System parameters with non-default values:
processes = 80
sessions = 160
event = 10191 trace name context forever, level 1
sga_max_size = 1795162112
shared_pool_size = 872415232
shared_pool_reserved_size= 86888153
filesystemio_options = setall
control_files = /oracle/<SID>/origlogA/cntrl/cntrl<SID>.dbf, /oracle/<SID>/origlogB/cntrl/cntrl<SID>.dbf, /oracle/<SID>/sapdata1/cntrl/cntrl<SID>.dbf
control_file_record_keep_time= 30
db_block_size = 8192
db_cache_size = 872415232
compatible = 10.2.0
log_archive_dest = /oracle/<SID>/oraarch/<SID>arch
log_buffer = 14477312
log_checkpoints_to_alert = TRUE
dml_locks = 4000
replication_dependency_tracking= FALSE
undo_management = AUTO
undo_tablespace = PSAPUNDO
_in_memory_undo = FALSE
undo_retention = 43200
recyclebin = off
remote_os_authent = TRUE
remote_login_passwordfile= EXCLUSIVE
job_queue_processes = 1
parallel_execution_message_size= 16384
_table_lookup_prefetch_size= 0
background_dump_dest = /oracle/<SID>/saptrace/background
user_dump_dest = /oracle/<SID>/saptrace/usertrace
core_dump_dest = /oracle/<SID>/saptrace/background
db_name = <SID>
open_cursors = 2000
_sort_elimination_cost_ratio= 10
_b_tree_bitmap_plans = FALSE
star_transformation_enabled= true
query_rewrite_enabled = false
_index_join_enabled = FALSE
_optim_peek_user_binds = FALSE
pga_aggregate_target = 1158508707
_optimizer_mjc_enabled = FALSE
PMON started with pid=2, OS id=18599
PSP0 started with pid=3, OS id=18601
MMAN started with pid=4, OS id=18603
DBW0 started with pid=5, OS id=18605
LGWR started with pid=6, OS id=18607
CKPT started with pid=7, OS id=18609
SMON started with pid=8, OS id=18611
RECO started with pid=9, OS id=18613
CJQ0 started with pid=10, OS id=18615
MMON started with pid=11, OS id=18617
MMNL started with pid=12, OS id=18619
Wed Apr 09 18:04:41 IST 2014
ALTER DATABASE MOUNT
Wed Apr 09 18:04:45 IST 2014
Setting recovery target incarnation to 1
Wed Apr 09 18:04:45 IST 2014
Successful mount of redo thread 1, with mount id 76579065
Wed Apr 09 18:04:45 IST 2014
Database mounted in Exclusive Mode
Completed: ALTER DATABASE MOUNT
Wed Apr 09 18:05:13 IST 2014
alter database archivelog
Wed Apr 09 18:05:13 IST 2014
Completed: alter database archivelog
Wed Apr 09 18:05:20 IST 2014
alter database open
Wed Apr 09 18:05:20 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_dbw0_18605.trc:
ORA-01157: cannot identify/lock data file 20 - see DBWR trace file
ORA-01110: data file 20: '/oracle/<SID>/sapdata2/sr3711_7/sr3711.data7'
ORA-27046: file size is not a multiple of logical block size
Additional information: 1
ORA-1157 signalled during: alter database open...
I think the database is getting mounted but not opening because I tried to turn on archive log manually using sqlplus, everything worked fine till the last command of "alter database open"
Thanks
Aps
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
ORA-27046: file size is not a multiple of logical block size
This error indicates file size mismatch,as it could be by usage of current controlfile to start database restored with 2 months back data.(Mismatch b/w controlfile & data files size).Here i could be wrong but it may be the possibility.
I suggest you try to compare the size for faulty file sr3711_7/sr3711.data7 with the correct one.If any difference restore this single file in place and follow the previous procedure to open database. Also suggesting a workaround for your last ref. ORA-27046 file size is not a multiple of logical block size | The Gruff DBA
Regards,
Gaurav
Hi Aps,
Could you please share alert_SID.log file also.
Regards,
Gaurav
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi RB
as you suggested now the Error has changed, with the oracle intialisation in progress
trans.log
-------------------------
R3trans -d
This is R3trans version 6.19 (release 711 - 26.10.11 - 13:00:00).
unicode enabled version
2EETW169 no connect possible: "DBMS = ORACLE --- dbs_ora_tnsname = '<SID>'"
R3trans finished (0012).
<hostname>:<SID>adm 22> more trans.log
4 ETW000 R3trans version 6.19 (release 711 - 26.10.11 - 13:00:00).
4 ETW000 unicode enabled version
4 ETW000 ===============================================
4 ETW000
4 ETW000 date&time : 09.04.2014 - 16:57:13
4 ETW000 control file: <no ctrlfile>
4 ETW000 R3trans was called as follows: R3trans -d
4 ETW000 trace at level 2 opened for a given file pointer
4 ETW000 [ dev trc,00000] Wed Apr 9 16:57:13 2014 573 0.000573
4 ETW000 [ dev trc,00000] db_con_init called 55 0.000628
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 168 0.000796
4 ETW000 [ dev trc,00000] create_con (con_name=R/3) 71 0.000867
4 ETW000 [ dev trc,00000] Loading DB library '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' ...
4 ETW000 156 0.001023
4 ETW000 [ dev trc,00000] DlLoadLib() success: dlopen("/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so"), hdl 0
4 ETW000 47825 0.048848
4 ETW000 [ dev trc,00000] Library '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' loaded 66 0.048914
4 ETW000 [ dev trc,00000] function DbSlExpFuns loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 173 0.049087
4 ETW000 [ dev trc,00000] Version of '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' is "710.21", patchlevel (0.146)
4 ETW000 492 0.049579
4 ETW000 [ dev trc,00000] function dsql_db_init loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 89 0.049668
4 ETW000 [ dev trc,00000] function dbdd_exp_funs loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 291 0.049959
4 ETW000 [ dev trc,00000] New connection 0 created 131 0.050090
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = -000000001, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 00
0, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 99 0.050189
4 ETW000 [ dev trc,00000] db_con_connect (con_name=R/3) 47 0.050236
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 69 0.050305
4 ETW000 [ dev trc,00000] find_con_by_name found the following connection: 44 0.050349
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = 000000000, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 000
, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 254 0.050603
4 ETW000 [ dev trc,00000] -->DbSlConnect(vers=-1, type=0, env='[default env]', userid='[default usr]')
4 ETW000 294 0.050897
4 ETW000 [ dev trc,00000] Register application info. 763 0.051660
4 ETW000 [ dev trc,00000] Oracle client version: 10.2.0.1.0, (dbsl 711 131011, UNICODE[2]) 181 0.051841
4 ETW000 [ dev trc,00000] Default connection: tnsname ='<SID>' 101 0.051942
4 ETW000 [ dev trc,00000] -->oci_initialize(con=0, char='UTF16', nchar='AL16UTF16', nls=-1),uc_ln=2
4 ETW000 122 0.052064
4 ETW000 [ dev trc,00000] OCIEnvCreate(mode=16384=OCI_UTF16) returned 0 (for default NLS handle)
4 ETW000 14429 0.066493
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrHp=60000000022455b0) 242 0.066735
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrBt=6000000002256168) 64 0.066799
4 ETW000 [ dev trc,00000] NLS env. settings: lang=0041004D00450052004900430041004E 707 0.067506
4 ETW000 [ dev trc,00000] terr=0041004D00450052004900430041 77 0.067583
4 ETW000 [ dev trc,00000] char=00550054004600310036 64 0.067647
4 ETW000 [ dev trc,00000] Client NLS setting (by OCINlsGetInfo(con=0)): 'AMERICAN_AMERICA.UTF16'
4 ETW000 76 0.067723
4 ETW000 [ dev trc,00000] charset='UTF16', ncharset='AL16UTF16', UNI_ASC=FALSE 52 0.067775
4 ETW000 [ dev trc,00000] Logon as OPS$-user to get SAPSR3's password 144 0.067919
4 ETW000 [ dev trc,00000] Connecting as /@<SID> on connection 0 (nls 0) ... (dbsl 711 131011, UNICODE[2])
4 ETW000 84 0.068003
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_SVCCTX, con=0, svchp=6000000002256098) 135 0.068138
4 ETW000 [ dev trc,00000] Allocating server context handle 56 0.068194
4 ETW000 [ dev trc,00000] Attaching to database server <SID> (con=0, svchp=6000000002256098, srvhp=60000000022598e8)
4 ETW000 90 0.068284
4 ETW000 [ dev trc,00000] Assigning server context 60000000022598e8 to service context 6000000002256098
4 ETW000 83670 0.151954
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_SESSION, con=0, usrhp=6000000002245dc8) 311 0.152265
4 ETW000 [ dev trc,00000] Starting user session: OCISessionBegin(con=0, usr='/', svc=6000000002256098, srv=60000000022598e8, usr=6000000002245dc8)
4 ETW000 86 0.152351
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with -1=OCI_ERROR 14410 0.166761
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with SQL error 1033: 389 0.167150
4 ETW000 [ dev trc,00000] ORA-01033: ORACLE initialization or shutdown in progress
4 ETW000 289 0.167439
4 ETW000 [ dev trc,00000] server_detach(con=0,stale=1,srvhp=60000000022598e8) 57 0.167496
4 ETW000 [ dev trc,00000] Server handle (con=0,svchp=6000000002256098,srvhp=60000000022598e8) detached
4 ETW000 5009 0.172505
4 ETW000 [ dev trc,00000] Free server handle 60000000022598e8 of con=0 54 0.172559
4 ETW000 [ dbsloci.,00000] *** ERROR => CONNECT failed with sql error '1033' 181 0.172740
4 ETW000 [ dev trc,00000] set_ocica() -> SQL error code 1033 164 0.172904
4 ETW000 [ dev trc,00000] -->oci_get_errmsg (con=0, rc=1033) 58 0.172962
4 ETW000 [ dev trc,00000] OCIErrorGet -> SQL error code: 1033 161 0.173123
4 ETW000 [ dev trc,00000] ORA-01033: ORACLE initialization or shutdown in progress
4 ETW000 75 0.173198
4 ETW000 [ dev trc,00000] SAPUSER error or connect error 99=DBSL_ERR_DB, oerr=1033. 54 0.173252
4 ETW000 [ dev trc,00000] DbSlConnect(con=0) -> orc=0, 99=DBSL_ERR_DB 55 0.173307
4 ETW000 [ dblink ,00000] ***LOG BY2=>sql error 1033 performing CON 404 0.173711
4 ETW000 [ dblink ,00000] ***LOG BY0=>ORA-01033: ORACLE initialization or shutdown in progress
4 ETW000 84 0.173795
2EETW169 no connect possible: "DBMS = ORACLE --- dbs_ora_tnsname = '<SID>'"
-------------------------
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with -1=OCI_ERROR 14410 0.166761
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with SQL error 1033: 389 0.167150
4 ETW000 [ dev trc,00000] ORA-01033: ORACLE initialization or shutdown in progress
Now the error is not related to the listener.
su - orasid
sqlplus "/as sysdba"
select name, open_mode from v$database;
If the database is down then start the database.
Regards
RB
Hi RB
I did what u said but this is what I get when I run "startup"
-------------------
SQL> startup
ORACLE instance started.
Total System Global Area 1795162112 bytes
Fixed Size 2068648 bytes
Variable Size 905973592 bytes
Database Buffers 872415232 bytes
Redo Buffers 14704640 bytes
Database mounted.
ORA-01157: cannot identify/lock data file 20 - see DBWR trace file
ORA-01110: data file 20: '/oracle/<SID>/sapdata2/sr3711_7/sr3711.data7'
----------------------------------------------
Hello,
This means the datafile /oracle/<SID>/sapdata2/sr3711_7/sr3711.data7 does not exist, or the RDBMS user does not have the neede ACL to access it.
It should be a file whose proprietary is orasid:sapsys.
You can set the ACL to 664 on the datafiles.
Can you :
- check you restored the datafiles in the reight location.
- check the access rights on the files ?
Best regards,
Steve.
HI Deepak, Gaurav,
No the control files are there in all the 3 locations, and I did stop the listener and start again the listener, but no luck.
The same error.
The control files are from todays date, and they are present.
Thanks
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
What about controlfile ?
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
HI Folks, my DC team has restored the /oracle/SID/102_64 mount point and the sapdata files are from the old back up from 2 month old.
I can connect to sqlplus as sysdba which I was not able to do before.
I have started the listener using listener start but I cannot startsap using the startsap command. Folks I am attaching the contents of the trans.log file which says that the listener is not up. help is apppreciated.
----------------------------
more trans.log
4 ETW000 R3trans version 6.19 (release 711 - 26.10.11 - 13:00:00).
4 ETW000 unicode enabled version
4 ETW000 ===============================================
4 ETW000
4 ETW000 date&time : 08.04.2014 - 15:05:27
4 ETW000 control file: <no ctrlfile>
4 ETW000 R3trans was called as follows: R3trans -d
4 ETW000 trace at level 2 opened for a given file pointer
4 ETW000 [ dev trc,00000] Tue Apr 8 15:05:27 2014 528 0.000528
4 ETW000 [ dev trc,00000] db_con_init called 51 0.000579
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 143 0.000722
4 ETW000 [ dev trc,00000] create_con (con_name=R/3) 74 0.000796
4 ETW000 [ dev trc,00000] Loading DB library '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' ...
4 ETW000 158 0.000954
4 ETW000 [ dev trc,00000] DlLoadLib() success: dlopen("/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so"), hdl 0
4 ETW000 68452 0.069406
4 ETW000 [ dev trc,00000] Library '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' loaded 78 0.069484
4 ETW000 [ dev trc,00000] function DbSlExpFuns loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 177 0.069661
4 ETW000 [ dev trc,00000] Version of '/usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so' is "710.21", patchlevel (0.146)
4 ETW000 497 0.070158
4 ETW000 [ dev trc,00000] function dsql_db_init loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 83 0.070241
4 ETW000 [ dev trc,00000] function dbdd_exp_funs loaded from library /usr/sap/<SID>/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 319 0.070560
4 ETW000 [ dev trc,00000] New connection 0 created 118 0.070678
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = -000000001, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 00
0, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 97 0.070775
4 ETW000 [ dev trc,00000] db_con_connect (con_name=R/3) 46 0.070821
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 65 0.070886
4 ETW000 [ dev trc,00000] find_con_by_name found the following connection: 43 0.070929
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = 000000000, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 000
, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 253 0.071182
4 ETW000 [ dev trc,00000] -->DbSlConnect(vers=-1, type=0, env='[default env]', userid='[default usr]')
4 ETW000 283 0.071465
4 ETW000 [ dev trc,00000] Register application info. 735 0.072200
4 ETW000 [ dev trc,00000] Oracle client version: 10.2.0.1.0, (dbsl 711 131011, UNICODE[2]) 192 0.072392
4 ETW000 [ dev trc,00000] Default connection: tnsname ='<SID>' 97 0.072489
4 ETW000 [ dev trc,00000] -->oci_initialize(con=0, char='UTF16', nchar='AL16UTF16', nls=-1),uc_ln=2
4 ETW000 107 0.072596
4 ETW000 [ dev trc,00000] OCIEnvCreate(mode=16384=OCI_UTF16) returned 0 (for default NLS handle)
4 ETW000 15088 0.087684
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrHp=60000000022455b0) 238 0.087922
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrBt=6000000002256168) 64 0.087986
4 ETW000 [ dev trc,00000] NLS env. settings: lang=0041004D00450052004900430041004E 752 0.088738
4 ETW000 [ dev trc,00000] terr=0041004D00450052004900430041 80 0.088818
4 ETW000 [ dev trc,00000] char=00550054004600310036 64 0.088882
4 ETW000 [ dev trc,00000] Client NLS setting (by OCINlsGetInfo(con=0)): 'AMERICAN_AMERICA.UTF16'
4 ETW000 76 0.088958
4 ETW000 [ dev trc,00000] charset='UTF16', ncharset='AL16UTF16', UNI_ASC=FALSE 53 0.089011
4 ETW000 [ dev trc,00000] Logon as OPS$-user to get SAPSR3's password 138 0.089149
4 ETW000 [ dev trc,00000] Connecting as /@<SID> on connection 0 (nls 0) ... (dbsl 711 131011, UNICODE[2])
4 ETW000 84 0.089233
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_SVCCTX, con=0, svchp=6000000002256098) 129 0.089362
4 ETW000 [ dev trc,00000] Allocating server context handle 56 0.089418
4 ETW000 [ dev trc,00000] Attaching to database server <SID> (con=0, svchp=6000000002256098, srvhp=60000000022598e8)
4 ETW000 86 0.089504
4 ETW000 [ dev trc,00000] OCIServerAttach(OCI_DEFAULT) failed with -1=OCI_ERROR 17668 0.107172
4 ETW000 [ dev trc,00000] OCIServerAttach(OCI_DEFAULT) failed with SQL error 12541: 417 0.107589
4 ETW000 [ dev trc,00000] ORA-12541: TNS:no listener
4 ETW000 104 0.107693
4 ETW000 [ dbsloci.,00000] *** ERROR => CONNECT failed with sql error '12541' 93 0.107786
4 ETW000 [ dev trc,00000] set_ocica() -> SQL error code 12541 51 0.107837
4 ETW000 [ dev trc,00000] -->oci_get_errmsg (con=0, rc=12541) 58 0.107895
4 ETW000 [ dev trc,00000] OCIErrorGet -> SQL error code: 12541 155 0.108050
4 ETW000 [ dev trc,00000] ORA-12541: TNS:no listener
4 ETW000 87 0.108137
4 ETW000 [ dev trc,00000] SAPUSER error or connect error 99=DBSL_ERR_DB, oerr=12541. 55 0.108192
4 ETW000 [ dev trc,00000] DbSlConnect(con=0) -> orc=0, 99=DBSL_ERR_DB 55 0.108247
4 ETW000 [ dblink ,00000] ***LOG BY2=>sql error 12541 performing CON 375 0.108622
4 ETW000 [ dblink ,00000] ***LOG BY0=>ORA-12541: TNS:no listener 79 0.108701
2EETW169 no connect possible: "DBMS = ORACLE --- dbs_ora_tnsname = '<SID>'"
-----------------------
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
4 ETW000 [ dev trc,00000] OCIServerAttach(OCI_DEFAULT) failed with -1=OCI_ERROR 17668 0.107172
4 ETW000 [ dev trc,00000] OCIServerAttach(OCI_DEFAULT) failed with SQL error 12541: 417 0.107589
4 ETW000 [ dev trc,00000] ORA-12541: TNS:no listener
su - orasid
lsnrctl stop
lsnrctl start
exit
su - sidadm
R3trans -d
and now provide the trans.log
Regards
RB
Thanks to you all
@RB - I know I have asked a lot of senseless questions but I think I just freaked out more than required.
@Nick Loy - thanks right now DC team restoring back up 2 months old on to the /oracle/SID/102_64. and then I will create the control file stuff. Let you know after that.
I will update the blog after this is resolved, thanks to everybody else who provided the information I really appreciate this.
Thanks
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Gaurav,
At this point I do not care what data is there on the server I just want it running.
If I do a system refresh will I be able to restore the Server or no?
What I want to know is that will it work or no, thanks man.
Regards,
Aps
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
Unfortunately it's no (If go with refresh activity with DEV or PRD). But after refresh you can restore the system with your previous 2 months backup data with you.Follow System Copy guides for this activity.
In refresh activity you need to install oracle 11g (Procedure is like you are going to install a fresh system but using data from other system) .I suggest from the existing system just backup required files like complete trans directory. .
Regards,
Gaurav
Hi Aps,
If data is not important as you have mentioned, you can restore dev or prd, however install oracle first.
Prd because it can be useful if you want to test on almost real time data.
Follow the System refresh procedure, like backup and restore etc to complete the refresh.
Before starting anything now, first take the backup of the file system.
For the oracle issues, re-install the oracle software. Apply the latest SBP too.
1525025 - Oracle 11.2.0: Oracle Software Deinstallation
Regards, Jatin
Hi Reagan, Gaurav and Deepak
I have the control files but I think they are from 2 month old backup.
I did run the commands as reagan sugggested on the Dev and the Qua server.I coudnot count the files on both the servers but definately the files on the DEV server are more in count.
What the hell am I suppose to do. Shall I install oracle from a fresh if yes then how do I go about it.
Shall I fiirst deintall Oracle 10g and SAP ?
Shall I deintall oracle 10g only ?
Shall I start intalling oracle on the system with the old one present as well and the new oracle is going to verwrite the old Oracle.?
I have done Oracle Upgrade recently using OUI from 10g to 11g on both windows and UNIX but how do I go about fresh installation.
I need help folks
Regards,
Aps
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
First what is your approach now,are you going with old backup restore or going with system refresh either by PRD or by DEV.Please confirm.For your query
Shall I fiirst deintall Oracle 10g and SAP ?
If you are going for fresh installation try to clear oracle & sap.Also you mentioned that you recent upgraded the oracle 10g to 11g.So go with 11g installation instead of 10g for fresh installation.
Regards,
Gaurav
Hi Aps,
If I do a system refresh will I be able to restore the Server or no?
If you go with system refresh you would not be able to restore your existing one as it'll refresh this system either by PRD or by DEV data.
If I go for a fresh Installtion can you provide me with some notes or I just follow the installation guide for Installing SAP PI ?
Better to follow standard,general SAP PI installation guides.
Regards,
Gaurav
What the hell am I suppose to do. Shall I install oracle from a fresh if yes then how do I go about it.
Read Oracle installation guide.
Shall I fiirst deintall Oracle 10g and SAP ?
Shall I deintall oracle 10g only ?
Shall I start intalling oracle on the system with the old one present as well and the new oracle is going to verwrite the old Oracle.?
I have done Oracle Upgrade recently using OUI from 10g to 11g on both windows and UNIX but how do I go about fresh installation.
Too many irrelevant and baseless questions. Why do you want to uninstall SAP ?
As this is an Unix system you don't need to uninstall the Oracle or SAP. Just delete the files under $ORACLE_HOME and start the Oracle installation.
You don't need to upgrade the database if you are installing the database software. Install the database version which was used before and apply the latest SBP.
Adapt the environment variables if required and start SAP.
Regards
RB
if the command was executed under /oracle/SID then yo might have lost the environment variable files, and the files under /oracle/SID/112_64 including the parameter files. The database datafiles and redologs will be there. If one of the control files were deleted then copy from the other location (if possible). You probably need to re-install the Oracle software to fix this.
I would first try to copy the missing files from another server in the landscape.
You can try to identify the missing files by modifying your command by replacing the rm condition with the listing condition and then execute the command on both the servers and find the missing ones.
Regards
RB
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Josef and Gaurav,
@josef I was logged in as /oracle/SID when I eneterd the command
@ Gaurav I do not mind doing the complete restore, but I think I have control files from the old back up and also I do have the sapdata files.
My question to you all is that If I do a complete Restore will the oracle binaries be restored or not and do I need to follow some procedures that I need to do after this is complete.
Regards
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
Oracle binaries are not backed up as a part of oracle database backup. You need to backup them separately using file system backup.
Worst case scenario.
Take a copy of oracle home from PRD or DEV and paste it into QAS.
Adjust the listener.ora, tnsnames.ora and sqlnet.ora files.
Post this restore the fresh backup from production and recover it.
Start the database and then application.
Hope this helps.
Regards,
Deepak Kori
Hi Aps,
My question to you all is that If I do a complete Restore will the oracle binaries be restored or not and do I need to follow some procedures that I need to do after this is complete.
If you do a complete restore oracle binaries will not be restored from the backup (Usually we not take backup of oracle binaries,just confirm it with your database team),for this you need to re install oracle 10g software as said earlier.
Also please clear for sapdata files,(these files are from latest or also from the old)
I think I have control files from the old back up and also I do have the sapdata files.
Regards,
Gaurav
find . -type f -mtime +15 -exec rm -f {} \;
Good news is:
Data files should still be there, because they should be younger than 15 days.
No need to restore database backup imho.
You may have lost control files though. Please first check if at least one ot them is left!
And you still haven't told us:
What was your current directory when you typed the command above?
PS
And abou the 'disk full' messages:
Check if you lost mount points or filesystems!
PPS:
And if you already startet restore, my advice from above may be too late. Sorry for that.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi,
Since you have datafiles, you can recover your database as below:
1) rsync the Oracle home between your production and Quality (/oracle/SID/102_64)
2) Create controlfile and recover/open database
Other easiest option I would follow:
1) rsync the Oracle home same as above
2) Restore Production database into Quality (It will be your Quality refresh)
This option will give me the consistent system with latest production data
*All other folders can be checked/corrected manually (for sap*)
Regards,
Nick Loy
Hi Deepak and Gaurav,
I have complete offline backup on Tape, I can use that. But how do I check the santity of the DB (Oracle).
I do have enough space on the orarch, only saptrace and sapdata2 are full.
Guys I did tell the DC team to restore the Quality server from the old backup but the Restore failed on the following drives i.e.
1) /oracle/sid/oraarch - disk full
2) /oracle/sid/saptrace - disk full
3) /oracle/sid/102_64 - disk full
4) /oracle/sid/sapdata2 - disk full
After the delete debacle I saw that there was no sqlplus in the /oracle/xqa so I thought of I restore but that did not go well as you can see.
So what do I do now, shall I tell them to do it again anything specific that I need to tell them.
Regards.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
If you have valid backup then no worries,also as you mentioned you're not able to connect database via command sqlplus "/as sysdba" (As per me during deletion some database executable's also get removed from the system.Try to find out first what things left & what are gone completely.
If we're going for complete restore we can go for installation up to database load phase & here we can restore our old backup from tapes.
Regards,
Gaurav
Hi Aps,
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:36 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
ARC1 started with pid=35, OS id=26204
After issue occurred have you restored some data files ? Something changed / misplaced (Which is already with your environment.)
Try to restore it from valid backup.
Regards,
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Gaurav following are the logs:
last 300 lines of alert_SID.log File contents are
-------------------------------
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26200.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:36 IST 2014
ARCH: Detected ARCH process failure
ARCH: Detected ARCH process failure
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=14, OS id=26202
Mon Apr 07 12:00:36 IST 2014
ARC0: Archival started
ARC1: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 07 12:00:36 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:36 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
ARC1 started with pid=35, OS id=26204
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:36 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:41 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:41 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:41 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:41 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26202.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:43 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26204.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:45 IST 2014
ARCH: Detected ARCH process failure
ARCH: Detected ARCH process failure
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=14, OS id=26206
Mon Apr 07 12:00:45 IST 2014
ARC0: Archival started
ARC1: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 07 12:00:45 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:45 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
ARC1 started with pid=36, OS id=26208
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:45 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:50 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:50 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:51 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:51 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:52 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:52 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:52 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26208.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:53 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:53 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:53 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26206.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:54 IST 2014
ARCH: Detected ARCH process failure
ARCH: Detected ARCH process failure
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=35, OS id=26211
Mon Apr 07 12:00:55 IST 2014
ARC0: Archival started
ARC1: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 07 12:00:55 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
ARC1 started with pid=36, OS id=26213
Mon Apr 07 12:00:55 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:00:55 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:59 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:00:59 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:00 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:00 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26213.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:01 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26211.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:04 IST 2014
ARCH: Detected ARCH process failure
ARCH: Detected ARCH process failure
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=14, OS id=26215
Mon Apr 07 12:01:04 IST 2014
ARC0: Archival started
ARC1: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
ARC1 started with pid=37, OS id=26217
Mon Apr 07 12:01:04 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:01:04 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:04 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:01:04 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:09 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:09 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:09 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:09 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:10 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:10 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:10 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26217.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:11 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:11 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:11 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26215.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:13 IST 2014
ARCH: Detected ARCH process failure
ARCH: Detected ARCH process failure
ARCH: STARTING ARCH PROCESSES
ARC0 started with pid=35, OS id=26219
Mon Apr 07 12:01:13 IST 2014
ARC0: Archival started
ARC1: Archival started
ARCH: STARTING ARCH PROCESSES COMPLETE
Mon Apr 07 12:01:13 IST 2014
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:01:13 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26219.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
ARC1 started with pid=36, OS id=26221
Error: Controlfile was changed externally while mounted
Please check if another Oracle database is running
and accessing the same controlfile
Mon Apr 07 12:01:13 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc1_26221.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:20 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26219.trc:
ORA-00600: internal error code, arguments: [2141], [67556390], [60878963], [], [], [], [], []
Mon Apr 07 12:01:20 IST 2014
Errors in file /oracle/<SID>/saptrace/background/<SID>_arc0_26219.trc:
ORA-00600: internal error code,
Contents of trans.log
---------------------------------
more trans.log
4 ETW000 R3trans version 6.19 (release 711 - 26.10.11 - 13:00:00).
4 ETW000 unicode enabled version
4 ETW000 ===============================================
4 ETW000
4 ETW000 date&time : 07.04.2014 - 15:47:54
4 ETW000 control file: <no ctrlfile>
4 ETW000 R3trans was called as follows: R3trans -d
4 ETW000 trace at level 2 opened for a given file pointer
4 ETW000 [ dev trc,00000] Mon Apr 7 15:47:54 2014 671 0.000671
4 ETW000 [ dev trc,00000] db_con_init called 48 0.000719
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 173 0.000892
4 ETW000 [ dev trc,00000] create_con (con_name=R/3) 86 0.000978
4 ETW000 [ dev trc,00000] Loading DB library '/usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so' ...
4 ETW000 217 0.001195
4 ETW000 [ dev trc,00000] Mon Apr 7 15:47:55 2014 46903 0.048098
4 ETW000 [ dev trc,00000] DlLoadLib() success: dlopen("/usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so"), hdl 0
4 ETW000 841 0.048939
4 ETW000 [ dev trc,00000] Library '/usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so' loaded 66 0.049005
4 ETW000 [ dev trc,00000] function DbSlExpFuns loaded from library /usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 183 0.049188
4 ETW000 [ dev trc,00000] Version of '/usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so' is "710.21", patchlevel (0.146)
4 ETW000 569 0.049757
4 ETW000 [ dev trc,00000] function dsql_db_init loaded from library /usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 92 0.049849
4 ETW000 [ dev trc,00000] function dbdd_exp_funs loaded from library /usr/sap/SID/SYS/exe/uc/hpia64/dboraslib.so
4 ETW000 350 0.050199
4 ETW000 [ dev trc,00000] New connection 0 created 150 0.050349
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = -000000001, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 00
0, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 102 0.050451
4 ETW000 [ dev trc,00000] db_con_connect (con_name=R/3) 47 0.050498
4 ETW000 [ dev trc,00000] determine_block_commit: no con_hdl found as blocked for con_name = R/3
4 ETW000 72 0.050570
4 ETW000 [ dev trc,00000] find_con_by_name found the following connection: 44 0.050614
4 ETW000 [ dev trc,00000] 0: name = R/3, con_id = 000000000, state = DISCONNECTED, tx = NO , bc = NO , hc = NO , perm = YES, reco = NO , frco = NO , timeout = 000
, con_max = 255, con_opt = 255, occ = NO , prog =
4 ETW000 262 0.050876
4 ETW000 [ dev trc,00000] -->DbSlConnect(vers=-1, type=0, env='[default env]', userid='[default usr]')
4 ETW000 294 0.051170
4 ETW000 [ dev trc,00000] Register application info. 803 0.051973
4 ETW000 [ dev trc,00000] Oracle client version: 10.2.0.1.0, (dbsl 711 131011, UNICODE[2]) 197 0.052170
4 ETW000 [ dev trc,00000] Default connection: tnsname ='SID' 61 0.052231
4 ETW000 [ dev trc,00000] -->oci_initialize(con=0, char='UTF16', nchar='AL16UTF16', nls=-1),uc_ln=2
4 ETW000 120 0.052351
4 ETW000 [ dev trc,00000] OCIEnvCreate(mode=16384=OCI_UTF16) returned 0 (for default NLS handle)
4 ETW000 8016 0.060367
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrHp=60000000022455b0) 207 0.060574
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_ERROR, con=0, ErrBt=6000000002256168) 65 0.060639
4 ETW000 [ dev trc,00000] NLS env. settings: lang=0041004D00450052004900430041004E 366 0.061005
4 ETW000 [ dev trc,00000] terr=0041004D00450052004900430041 78 0.061083
4 ETW000 [ dev trc,00000] char=00550054004600310036 86 0.061169
4 ETW000 [ dev trc,00000] Client NLS setting (by OCINlsGetInfo(con=0)): 'AMERICAN_AMERICA.UTF16'
4 ETW000 75 0.061244
4 ETW000 [ dev trc,00000] charset='UTF16', ncharset='AL16UTF16', UNI_ASC=FALSE 53 0.061297
4 ETW000 [ dev trc,00000] Logon as OPS$-user to get SAPSR3's password 153 0.061450
4 ETW000 [ dev trc,00000] Connecting as /@SID on connection 0 (nls 0) ... (dbsl 711 131011, UNICODE[2])
4 ETW000 85 0.061535
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_SVCCTX, con=0, svchp=6000000002256098) 174 0.061709
4 ETW000 [ dev trc,00000] Allocating server context handle 66 0.061775
4 ETW000 [ dev trc,00000] Attaching to database server SID (con=0, svchp=6000000002256098, srvhp=60000000022598e8)
4 ETW000 93 0.061868
4 ETW000 [ dev trc,00000] Assigning server context 60000000022598e8 to service context 6000000002256098
4 ETW000 983608 1.045476
4 ETW000 [ dev trc,00000] OCIHandleAlloc(OCI_HTYPE_SESSION, con=0, usrhp=6000000002245dc8) 192 1.045668
4 ETW000 [ dev trc,00000] Starting user session: OCISessionBegin(con=0, usr='/', svc=6000000002256098, srv=60000000022598e8, usr=6000000002245dc8)
4 ETW000 87 1.045755
4 ETW000 [ dev trc,00000] Mon Apr 7 15:47:56 2014 35897 1.081652
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with -1=OCI_ERROR 138 1.081790
4 ETW000 [ dev trc,00000] OCISessionBegin(OCI_DEFAULT) failed with SQL error 257: 404 1.082194
4 ETW000 [ dev trc,00000] ORA-00257: archiver error. Connect internal only, until freed.
4 ETW000 100 1.082294
4 ETW000 [ dev trc,00000] server_detach(con=0,stale=1,srvhp=60000000022598e8) 65 1.082359
4 ETW000 [ dev trc,00000] Server handle (con=0,svchp=6000000002256098,srvhp=60000000022598e8) detached
4 ETW000 1122 1.083481
4 ETW000 [ dev trc,00000] Free server handle 60000000022598e8 of con=0 194 1.083675
4 ETW000 [ dbsloci.,00000] *** ERROR => CONNECT failed with sql error '257' 186 1.083861
4 ETW000 [ dev trc,00000] set_ocica() -> SQL error code 257 88 1.083949
4 ETW000 [ dev trc,00000] -->oci_get_errmsg (con=0, rc=257) 64 1.084013
4 ETW000 [ dev trc,00000] OCIErrorGet -> SQL error code: 257 160 1.084173
4 ETW000 [ dev trc,00000] ORA-00257: archiver error. Connect internal only, until freed.
4 ETW000 87 1.084260
4 ETW000 [ dev trc,00000] SAPUSER error or connect error 99=DBSL_ERR_DB, oerr=257. 63 1.084323
4 ETW000 [ dev trc,00000] DbSlConnect(con=0) -> orc=0, 99=DBSL_ERR_DB 57 1.084380
4 ETW000 [ dblink ,00000] ***LOG BY2=>sql error 257 performing CON 557 1.084937
4 ETW000 [ dblink ,00000] ***LOG BY0=>ORA-00257: archiver error. Connect internal only, until freed.
4 ETW000 88 1.085025
2EETW169 no connect possible: "DBMS = ORACLE --- dbs_ora_tnsname = 'SID'"
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Deepak and Gaurav,
The Database is not working properly when I try to login as sqlplus"/as sysdba"
it is asking me for a username and password which is not how it should be working and somehow the /oracle/sid/saptrace mount point is getting 100% no matter what I do.
also R3trans -d is giving a code 12 which is obvious since the DB is not working. Listeneer is up though but I cannot start the DB since I cannot loginto sqlpus.
I cannot also change the password for any users since brconnect and BRTOOLS is failing as well.
If you want I can post some logs like the alert_SID.log and trans.log
Regards.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Deepak and Gaurav,
when I had given the command I within maybe 30 seconds pressed control+c and exited out but the damage was already done.
Now I have the complete offline backup from 2 months ago and It is by the grace of god not Production Environment, but it is Quality Server.
Yes Gaurav they use HP DATA Protector to do backup and restores.
So please what should I tell my DC team that what should I do, also I can even tell them to do a restore from the Dev Backup.
I want you to please tell me the process to Restore the System and get the PI environment working again.
Thanks
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi Aps,
Thank to GOD that it's not your PRD system,Now you can go with previous restore & other option is to make a system copy of QTY from Production System.( It's on you from which system you gonna make system copy,practically it would be PRD.)
Tell your database team to perform restore activities as per standard methods of HP Data Protector.
Thanks,
Gaurav
Hi,
Here only the database backup will not help as we do not know what all folders got deleted.
Can you first check whether your oracle home is consistent.
Compare the file count and folder count with that from Production .
In case the files are consistent, you may try to connect to database. If the database connection works fine, you may proceed with restoration of data from backup.
Hope this helps.
Regards,
Deepak Kori
Hi Aps,
I think It would be a horrible situation at your end after deletion of complete files under directory /oracle/<SID> .As per me it's difficult to retrieve the structure if you are not with a valid backup (Hope it's not a Production environment).
For deleted files recovery you need to search out for a third party solution,not sure it'll retrieve the complete data files.Try to search any recent backup and if not,then last is to restore the system with previous backup & provide the redo files for complete recovery up till now.
Thanks,
Gaurav
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
Hi ,
1) Have you delete everything under /oracle/SID or what is the current state of the system ?
2) What kind of backup is available for the system and time period of that backup.
3) Do you have logs available from the last backup until today ?
Regards,
Deepak Kori
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.
User | Count |
---|---|
80 | |
9 | |
9 | |
7 | |
7 | |
6 | |
6 | |
6 | |
5 | |
4 |
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.