PI Dual Stack Update: MAIN_NEWBAS/XPRAS_AIMMRG hangs

[j_bayrhammer]

Hello,

we're in update to SPS14 on PI 7.3 double stack system.

We tested already on a sandbox - there were  no problems in this phase.

I noticed phase MAIN_NEWBAS/XPRAS_AIMMRG ran longer than on sandbox.

Therefore I checked SM50 on instance 00 - no jobs are running.

I checked SM37 on instance 00 - I saw jobs RDDEXECL. I did a status check as adviced in note 1947874. After that both jobs where cancelled.

Nothing else happened, SUM remained in status MAIN_NEWBAS/XPRAS_AIMMRG is running.

So I tried to logon on instance 09 (shadow instance). This did  not work - I didn't get a login window. I checked instance via dpmon and can see all DIA workprocesses are full and running (see screenshot).

In instance 00 I can see two dumps with error "DBIF_REPO_SL_ERROR" and "SQL error 60 occurred when accessing program "~L_PROXY_SERVICE================HPC" and "Databse error text: "ORA-00060: deadlock detected while waiting for resource".

File stderr3 in shadow instance work directory gives information:

Thu Aug 11 15:16:14 2016

*** ERROR => WARNING: Concurrent Generation of PH=~XTAB=========================HTC [scprehdr.c   6281]

{root-id=57AC003F7A9A09B0E10080000A70D4CD}_{conn-id=00000000000000000000000000000000}_0

SUM is still in status running, but processes are hanging on shadow instance.

At the moment I don't know what to do now.

Should I restart shadow instance?

Should I restart SUM?

Should I restart Job?

Regards,
Julia

[Sriram2009]

Hi Julia.

Stop the SUM, do the full system restart and then try again in the same phase.

BR

SS

[j_bayrhammer]

Hello,

we now killed all DIA workprocesses on shadow instance (oldest first...). After a few minutes all DIA were free. SUM then went on with procedure. I don't know which consequences this will have. In my opinion SUM should notice or repair missing things. The cancelled RDDEXECL Jobs weren't restarted. We'll see...

Regards,

Julia

[j_bayrhammer]

From SAP we got information that there was probably a deadlock situation while generation of sources. Proposal was to restart shadow instance. After restart we should check if RDDEXECL is shown as cancelled. SUM should notice the abortion and should show a dialog for repeat.

In our case we killed the work processes before. After that SUM ran on.

Perhaps there was a mass compilation of programs which ran into dead lock situation. Killing processes cancelled the generation of programs. This can be repeated every time (says SAP). SUM continued. If SUM does not offer a repeat, so there will be no problem (says SAP).

Regards

Julia