503 service unavailable - cannot redispatch request: connection already closed

[Ganimede-Dignan]

Hi,

several time I see error message "503 service unavailable" on a NW 7.11 CE box.

In /usr/sap/MIP/JXX/work/dev_icm, when message is displayed I can see:

[Thr 05] Fri Jul  5 09:34:04 2013

[Thr 05] JNCMIHttpNodeDel: node 611119350 is removed from the load balancing list

[Thr 05] HttpJ2EELbDel: server 611119350 lost protocol HTTP

[Thr 05] *** ERROR => HttpJ2EECleanupCallback(id=18/7016): cannot redispatch request: connection already closed: rc=-15 [http_j2ee2_m 2038]

[Thr 05] *** ERROR => HttpJ2EECleanupCallback(id=1/6932): cannot redispatch request: connection already closed: rc=-15 [http_j2ee2_m 2038]

[Thr 05] *** ERROR => HttpJ2EECleanupCallback(id=37/7010): cannot redispatch request: connection already closed: rc=-15 [http_j2ee2_m 2038]

[Thr 05] *** ERROR => HttpJ2EECleanupCallback(id=31/7021): cannot redispatch request: connection already closed: rc=-15 [http_j2ee2_m 2038]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=17/7023): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=4/7024): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=6/6794): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=10/7026): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=23/7027): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=28/7028): no server alive to handle request [http_j2ee2_m 2175]

[Thr 05] *** WARNING => HttpJ2EEFinalizeCallback(id=2/7030): no server alive to handle request [http_j2ee2_m 2175]

have you got any advice?

Thank you.

[Former Member]

Check status of J2EE instance when issue arises. Is it up and running? Provide logs for dispatcher and server nodes.

[bxiv]

Did you setup a dedicated message/CI system for this system?

I recently had issues where it appeared the diag servers were mis-behaving due to one of the CI nodes having issues with sapcpe; I updated from 7.3 SP0 to 7.3 SP9 and the system was much more useful.

What SP level are you at, and what is the current SP level for your system?

[Ganimede-Dignan]

Hi,

>Did you setup a dedicated message/CI system for this system?

No

>What SP level are you at, and what is the current SP level for your system?

NW 7.11 CE SP6 & SAP MII 12.1 SP9

[Former Member]

Hi,

it seems that the / all servernodes are restarted automatically.

there could be several reasons.

pls. have a look at the server logs at./usr/sap/MIP/JXX/j2ee\cluster\server0\log\system

[Ganimede-Dignan]

Something like that?

#2.0.#2013 07 05 09:49:53:152#+0200#Info#/System/Server#

com.sap.ASJ.ses.cf0011#BC-JAS-COR-SES#kernel.sda#00144F820CC900000000000400002305###com.sap.engine.core.session.Manager########Thread[main,5,main]#Plain##

Session user context persistency is set to DB#

#2.0.#2013 07 05 09:49:53:460#+0200#Info#/System/Server#

com.sap.ASJ.ses.cf0016#BC-JAS-COR-SES#kernel.sda#00144F820CC900000000000500002305###com.sap.engine.core.session.Manager########Thread[main,5,main]#Plain##

Session persistency storrage type is set to FILE#

#2.0.#2013 07 05 09:49:53:463#+0200#Info#/System/Server#

com.sap.ASJ.ses.cf0021#BC-JAS-COR-SES#kernel.sda#00144F820CC900000000000600002305###com.sap.engine.core.session.Manager########Thread[main,5,main]#Plain##

Session persistence mode is set to ON_REQUEST#

#2.0.#2013 07 05 09:49:55:174#+0200#Warning#/System/Server#

#BC-I18-JAV#tc~i18n~cp#00144F820CC900010000000000002305#611119350000000001##com.sap.i18n.cp.ConvertSimpleBase.Load shortcut failed#####7b40e879e54711e2b5fb00144f820cc9##0#Service Runner [tc~sec~destinations~p

rovider]#Plain##

Load shortcut failed. RscpInfo: Load shortcut failed.

InCp: 0100 OutCp: 0000

Rscp return code: 2048

Rscp error info:

.------------------------------============----------------------------.

|                              RSCP - Error                            |

| Error from:             Codepage handling (RSCP)                     |

| code: 2048  RSCPENOCONV  Wanted conversion not available.            |

| no shortcut conversion table                                         |

| module: rscpc    no:  259 line: 22167                    T100: TS012 |

| TSL01: CP9  p3: len=1,data=00                                        |

|             p4: (0100->4103): kind of T1 entry 536870912             |

|----------------------------------------------------------------------|

| Impromptu CCC initialization by                                      |

`----------------------------------------------------------------------'

#

#2.0.#2013 07 05 09:50:16:716#+0200#Info#/System/Server#

#BC-I18-JAV#tc~i18n~cpbase#00144F820CC900070000000000002305#611119350000000403##com.sap.i18n.cpbase.SAPLanguageInfo####7B4127DEE54711E2C10100144F820CC9#7b4127dee54711e2c10100144f820cc9#7b4127dee54711e2c101001

44f820cc9#0#Service Runner [I18NService]#Plain##

SAP Language-keys are not available from the backend System.In case of Double stack system please check if the destination 'I18NBackendConnection' is created.#

#2.0.#2013 07 05 09:50:16:747#+0200#Info#/System/Server#

#BC-I18-JAV#tc~i18n~saptimezone#00144F820CC900070000000100002305#611119350000000403##com.sap.i18n.SAPTimeZoneInfo####7B4127DEE54711E2C10100144F820CC9#7b4127dee54711e2c10100144f820cc9#7b4127dee54711e2c10100144

f820cc9#0#Service Runner [I18NService]#Plain##

SAPTimeZones are not available from the back end system.In case of Double stack system please check if the destination 'I18NBackendConnection' is created.#

09:49 should be the begin of the crash.

[Former Member]

hi,

hm, this is not the server.0.log? have also a look pls in the other logs at the destination

however are there additional languages installed?

pls Check TC I18n and then check .the parameter zcsa/installed_languages are all installed languages inserted?

[bxiv]

Have you maintained your timezone information correctly in the Java stack and the SQL server?

[Ganimede-Dignan]

Hi,

it's an Oracle box.... how can I set timezone in SAP J2EE and on Oracle rdbms?

Thank you.

[Ganimede-Dignan]

Hi,

>hm, this is not the server.0.log? have also a look pls in the other logs at the destination

I've already posted server.0.log

>however are there additional languages installed?

No, It's a J2EE only but this system call several ABAP backend Unicode system.

>pls Check TC I18n and then check .the parameter zcsa/installed_languages are all installed >languages inserted?

It's a J2EE only system, no TCODE I18n available.

Thank you,

[Reagan]

Hello

SAP picks up time from the OS level.

If the time settings were changed when the application was running then there will errors in the application.

Also if the server is configured with a domain controller for time synchronizing then make sure the time sync is maintained.

I believe a service called ntpd takes care of it

Good Luck

RB

[bxiv]

I believe Oracle follows the timezone settings on the server its installed to.

I think Java follows suit, however I know that the logs do not follow that logic; you can check with the config tool under the following path (I am reviewing based on a PI 7.11 install):

cluster-data > template - Usage_Type_All_in_One > Instance - ID######(Hostname) > VM Parameters tab > Additional tab > create a new entry with the following   -Duser.timezone and a value that is your timezone, for me its UTC.