Phase JOB_DBDIF_UPG failure in SAP470 110 -> ECC6.0 upgrade

[Former Member]

We are upgrading from SAP 470 110 to ECC6 .0 (Unix/Oracle)

Phase JOB_DBDIF_UPG fails when trying to run the report RADDBDIF in the background (in the "normal" instance, not the shadow)

The log PADDBDIF.LOG has the following entries:

1 ETQ201XEntering upgrade-phase "JOB_DBDIF_UPG" ("20071022114634")

2 ETQ367 Connect variables are set for standard instance access

4 ETQ399 System-nr = '10', GwService = 'sapgw10'

4 ETQ399 Environment variables:

4 ETQ399 dbs_ora_schema=SAPR3

4 ETQ399 auth_shadow_upgrade=0

4 ETQ260 Starting batchjob "RADDBDIF"

4 ETQ010 Date & Time: 20071022114634

4 ETQ260 Starting batchjob "RADDBDIF"

4 ETQ230 Starting RFC Login to: System = "DB1", GwHost = "hporap20",

GwService = "sapgw10"

4 ETQ359 RFC Login to: System="DB1", Nr="10", GwHost="hporap20",

GwService="sapgw10"

4 ETQ232 RFC Login succeeded

4 ETQ233 Calling function module "SUBST_START_BATCHJOB" by RFC

4 ETQ234 Call of function module "SUBST_START_BATCHJOB" by RFC succeeded

4 ETQ233 Calling function module "SUBST_CHECK_BATCHJOB" by RFC

4 ETQ010 Date & Time: 20071022114650

4 ETQ234 Call of function module "SUBST_CHECK_BATCHJOB" by RFC succeeded

4 ETQ239 Logging off from SAP system

1EETQ242 Error code "-1" during analysis of logfiles matching "RADDBDIF\.DB1"

2EETQ262 Batchjob "RADDBDIF" aborted

4 ETQ359 RFC Login to: System="DB1", Nr="10", GwHost="hporap20",

GwService="sapgw10"

4 ETQ232 RFC Login succeeded

2WETQ360 RFC of "subst_list_sys_logs" failed:

2WETQ361 code/exception : JOBLOG_READ_AND_DESIGN_ERROR

4 ETQ359 RFC Login to: System="DB1", Nr="10", GwHost="hporap20",

GwService="sapgw10"

4 ETQ232 RFC Login succeeded

1EETQ204 Upgrade phase "JOB_DBDIF_UPG" aborted with severe errors

("20071022114650")

RADDBDIF runs fine if run in in dialog (it even creates the log file /usr/sap/put/tmp/TADDBDIF.DB1) but fails when run from SM37.

Prior to this phase failing, I can see successful jobs in SM37 but when I try to look at the job logs I get an "error reading job log <log name>" message.

Within AL11 and at the UNIX level these files CAN be seen/modified by the db1adm user so this is NOT a permissions problem.

Any new SM37 jobs fail completely as they can't even write a log.

There is 1GB free space in the log directory and the same in /usr/sap/put so it is NOT a space problem.

Table TST03 has plenty of room for growth.

DIR_TRANS etc are all pointing to the correct locations and there are no application servers etc on this system.

The SAP kernel has been updated and SAPup etc were replaced with the latest versions at the start of the upgrade.

TEMSE consistency checks show no errors.

From SP11 I can see entries for the jobs prior to the phase failure but when I try to look at the contents I get the following error:

NAME: JOBLG<number>

RC: 2048

MSG: Type unknown

We did change the database codepage from US7ASCII to WE8DEC for the upgrade - not sure if this would cause this issue.

I'm running out of ideas and would appreciate any ideas on what I can do to progress this upgrade phase.

[markus_doehr2 · Accepted solution]

Any information in SM21 about where they fail to write the log?

--

Markus

[Former Member · Accepted solution]

Not really - there are 3 SM21 entries for each job failure:

message 1 - Type unknown

message 2 - Failed to create log for job RADDBDIF 233ÿÿÿÿÀ#####J#################/

message 3 - > Job RADDBDIF

Similar messages occur for all the standard housekeeping jobs etc. They don't really give any insight as to the cause of the error.

In ST11 all the DIR_XXX entries look fine and as mentioned in my post the log files for earlier SM37 jobs during the upgrade do actually exist within the OS but are unreadable.

Up until this phase failed the system was happily running background jobs and creating logs but all of a sudden is unable to create new logs or read old ones (from within SAP as at the OS level the db1adm user can read them, including from SM49 where I can run a "more" on the file and see the contents)

DIR_GLOBAL is /usr/sap/DB1/SYS/global (ie /sapmnt/DB1/global) as it has always been so has not been changed.

I have also checked the permissions on the folders UNDER the global directory (ie /sapmnt, /sapmnt/DB1 etc) but I can "work my way up" the directories one at a time as db1adm without a problem.

I should have mentioned I have also raised an OSS message for this but at this stage they are only asking fairly obvious questions, such as kernel levels, SAPup version etc which I have already covered in the message I raised with them.

[markus_doehr2 · Accepted solution]

Do the dev_w* traces for the background processes show any more light?

--

Markus

[Former Member · Accepted solution]

The only entry in the tracefile is

• ERROR => BtcEnvPrep: BtcLgOw failed: jobname=RADDBDIF , jobcount=14282100, rc=4 [btcjcntl.c 1460]

This only really tells me what I already know which is that RADDBDIF couldn't be run in background because it couldn't write a log (or least that is my interpretation of the message based on events)

[Former Member]

Hi,

Check your background workprocess availibility. and check the load at sm50.

Thanks,

[Former Member]

Hi all. My OSS note was escalated to Germany.

Taking into account the upgrade logs, TEMSE errors and SM21 error type (F4U), it seemed to point towards "data type table is incomplete / missing or there is a memory overwrite"

Table TST05 was empty so on their request I populated it with the following:

DTYPE DTHREAD DTCPUSP DTLANGS

BINARY

DATA X

OTFHPGL X X

OTFHPL2 X X

OTFLNPR X X

OTFLZPL X X

OTFPDF1 X X

OTFPOST X X

OTFPRES X X

OTFRDIF X X

OTFSTN2 X X

OTFSTND X X

OTFSWIN X X

OTFTELE X X

TEXT X X

TEXTHPGL X

TEXTHPLJ4 X X

TEXTHPLJIIID X X

TEXTKYOF1200 X X

TEXTKYOF12I2 X X

TEXTLN03 X X

TEXTLN03S X X

TEXTLN03SP X X

TEXTPOST2 X X

TEXTPOSTSCPT X X

TEXTWMF

This fixed everything - ie background jobs now run, the phase completed and I am now in the TABIM_UPG phase.

Many thanks to those who have have replied to this message I'm still unable to explain why this happened but hopefully this post may be of use to others in the future.

Regards

Dave

[Former Member]

Hi, Dave,

Did you retry to create that job? delete it and create it again and schedule it.

Thanks,

[Former Member]

I have tried to create new jobs, copied old jobs with different users etc. Same result.

While the problem seems to have been "activated" or "created" by one of the upgrade phases, it is now a "general" SAP system problem in that no background job, whether an upgrade job or a standard housekeeping job or a simple 1 line report, can run in background as job logs cannot be created or old logs read.

For example, I can run RSPO1043 (Spool consistency check) from SE38 in foreground but not as a background job.

The fact that I can no longer even read OLD job logs (ie a job ran successfully and wrote a log at the UNIX level 5 minutes before my upgrade ground to a halt but SAP can no longer read that log) is what is concerning me and pointing to some sort of issue with codepage settings or similar.

[Former Member]

Hi,

Change or check user authorization under which this is running.

thanks,

[Former Member]

It doesn't matter what user I try and run jobs under or read old job logs as - I get the same error. Users with SAP_ALL and SAP_NEW have exactly the same problem.

At the UNIX level the db1adm and oradb1 users do not appear to have changed - ie they are still members of the correct groups.

I even put oradb1 into the SAPSYS group in case the oradb1 user was trying to read the log files but it made no difference.