E-FILING GB - random error in RFC connections to GovernmentGateway

[gianluca_tacchella2]

Hello SCN experts,

we are trying to setup, using PI 7.11, the interface E-FILING for GB,

according to official SAP documentation:

https://service.sap.com/HRGB

http://help.sap.com/saphelp_erp60_sp/helpdata/en/0a/34e153038a424de10000000a174cb4/content.htm

and also according to the (very useful !!!) blog...

scn.sap.com/people/vasanthkumar.s2/blog/2009/03/19/e-filing-gb-configuration

Unfortunately we are facing a strange problem with the RFC destinations (Type G):

GB_EOY_XI

GB_XI_POLL

which try to contact respectively the following:

https://secure.gateway.gov.uk/submission

https://secure.gateway.gov.uk/poll

using port 443

The problem is that RANDOMLY they don´t work, which means that we have the following error during SM59 Connection Test:

ICM_HTTP_SSL_ERROR

Message no. SR000

Sometimes the Connection Test is not working, sometimes instead it works fine... giving back the response with status 'OK' '200'.

We can exclude that the problem is related to certificates,

because in STRUST we have the complete certificate chain according to OSS Note:

1420847 - New Government Gateway Security Certificates for eFiling

The ICM log (dev_icm) in case of ICM_HTTP_SSL_ERROR in SM59 Connection Test is:

[...]

[Thr 4412] MAIN TRACE BEGIN

[Thr 4412] IcmMsgProcess: Receive data from partner: WP(2), wp_no: 3

[Thr 4412] IcmRecMsg: received 2164 bytes

[Thr 4412] ============================================

[Thr 4412] | COM_DATA:

[Thr 4412] | Offset: 80 | Version: 7210

[Thr 4412] | MsgNo: 244 | Opcode: ICM_COM_OP_CONNECT (4)

[Thr 4412] ============================================

[Thr 4412] IcmHandleAdmMsg: op: 4, auth: 1

[Thr 4412] MPI<23e91>15#5 PeekSelectInbuf -1 0 202 (1) -> MPI_EOS: End Of Stream

[Thr 4412] IcmHandleAdmMsg: need new slot

[Thr 4412] NiBufDup: ref 1 for buf 00000000231044B0

[Thr 4412] IcmCreateRequest: Append request 350351

[Thr 4412] IcmQueueAppend: queuelen:     1

[Thr 4412] NiIWrite: hdl 217 sent data (wrt=104,pac=1,MESG_IO)

[Thr 4412] NiBufFree: ref 1 for buf 00000000231044B0

[Thr 4412] MAIN TRACE END

[Thr 7336] IcmWorkerThread: worker 1 got the semaphore

[Thr 7336] REQUEST:

    Type: CONNECT_TO_SERV    Index = 350351

[Thr 7336] IcmConnConnect: allocate new conn slot

[Thr 7336] IcmConnCheckStoredClientConn: next client timeout check in 59 sec

[Thr 7336] REQ TRACE BEGIN: 8/315129/1

[Thr 7336] MPI<23e91>15#6 PeekSelectInbuf -1 0 202 (1) -> MPI_EOS: End Of Stream

[Thr 7336] IcmConnAssignContext: searching for context:

[Thr 7336]     tid: 30, uid: 20900, mode: 1

[Thr 7336] IcmConnConnect: context 28 assigned to tid: 30, uid: 20900, mode: 1

[Thr 7336]     keep_alive_timeout: -1, proc_timeout: -1, wp_timeout: 500

[Thr 7336] NiIGetServNo: servicename '56301' = port 56301

[Thr 7336] IcmGetServicePtr: grpsapdx1.grouphc.net:56301 - new serv_ref_count: 1

[Thr 7336] IcmIConnConnect: direct connect to secure.gateway.gov.uk:443

[Thr 7336] NiHLGetNodeAddr: found hostname 'secure.gateway.gov.uk' in cache

[Thr 7336] NiIGetNodeAddr: hostname 'secure.gateway.gov.uk' = addr 157.203.50.169

[Thr 7336] NiIGetServNo: servicename '443' = port 443

[Thr 7336] NiICreateHandle: hdl 373 state NI_INITIAL_CON

[Thr 7336] NiIInitSocket: set default settings for new hdl 373/sock 20200 (I4; ST)

[Thr 7336] NiIBlockMode: set blockmode for hdl 373 FALSE

[Thr 7336] NiICheckPendConnection: connection of hdl 373 to 157.203.50.169:443 established

[Thr 7336] NiIConnect: hdl 373 took local address 10.70.52.28:56778

[Thr 7336] NiIConnect: state of hdl 373 NI_CONNECTED

[Thr 7336] <<- SapSSLSessionInit()==SAP_O_K

[Thr 7336]      in: args = "role=1 (CLIENT), auth_type=3 (USE_CLIENT_CERT)"

[Thr 7336]     out: sssl_hdl = 000000000E4C31B0

[Thr 7336] NiIBlockMode: set blockmode for hdl 373 TRUE

[Thr 7336]   SSL NI-sock: local=10.70.52.28:56778  peer=157.203.50.169:443

[Thr 7336] <<- SapSSLSetNiHdl(sssl_hdl=000000000E4C31B0, ni_hdl=373)==SAP_O_K

[Thr 7336]   SapISSLComposeFilename(): Filename = "G:\usr\sap\DX1\DVEBMGS63\sec\SAPSSLA.pse"

[Thr 7336] <<- SapSSLSetSessionCredential(sssl_hdl=000000000E4C31B0)==SAP_O_K

[Thr 7336]      in: cred_name = "G:\usr\sap\DX1\DVEBMGS63\sec\SAPSSLA.pse"

[Thr 7336] IcmConnInitClientSSL: using pse G:\usr\sap\DX1\DVEBMGS63\sec\SAPSSLA.pse, show client certificate if available

[Thr 7336] <<- SapSSLSetTargetHostname(sssl_hdl=000000000E4C31B0)==SAP_O_K

[Thr 7336]      in: hostname = "secure.gateway.gov.uk"

[Thr 7336] Wed Feb 18 14:28:28 2015

[Thr 7336] *** ERROR during SecudeSSL_SessionStart() from SSL_connect()==SSL_ERROR_CONNECTION_LOST

[Thr 7336]    session uses PSE file "G:\usr\sap\DX1\DVEBMGS63\sec\SAPSSLA.pse"

[Thr 7336] SecudeSSL_SessionStart: SSL_connect() failed --

[Thr 7336]   secude_error 4864 (0x00001300) = "ERROR recv(hdl=20200,buf=00000000000F4BB0,len=5)=-1, GetLastError()=10054 (0x00002746

[Thr 7336] >> ---------- Begin of Secude-SSL Errorstack ---------- >>

[Thr 7336] ERROR in BIO_read: (4864/0x1300) ERROR recv(hdl=20200,buf=00000000000F4BB0,len=5)=-1, GetLastError()=10054 (0x00002746) [Thr 7336] ERROR in sock_read: (4864/0x1300) ERROR recv(hdl=20200,buf=00000000000F4BB0,len=5)=-1,

GetLastError()=10054 (0x00002746)

[Thr 7336] << ---------- End of Secude-SSL Errorstack ----------

[Thr 7336]   SSL_get_state() returned 0x00002120 "SSLv3 read server hello A"

[Thr 7336]   No certificate request received from Server

[Thr 7336] <<- ERROR: SapSSLSessionStart(sssl_hdl=000000000E4C31B0)==SSSLERR_SSL_CONNECT

[Thr 7336] <<- SapSSLErrorName()==SSSLERR_SSL_CONNECT

[Thr 7336] *** ERROR => IcmConnInitClientSSL: SapSSLSessionStart failed (-57): SSSLERR_SSL_CONNECT {0008cef9} [icxxconn.c 1977]

[Thr 7336] <<- SapSSLSessionDone()==SAP_O_K

[Thr 7336]      in: sssl_hdl   = 000000000E4C31B0

[Thr 7336]          ... ni_hdl = 373

[Thr 7336] IcmConnConnect(id=8/315129): free MPI request blocks

[Thr 7336] MPI<23e91>15#7 GetInbuf -1 f28960 202 (1) -> MPI_EOS: End Of Stream

[Thr 7336] MPI<23e91>15#8 FreeInbuf#1 0 f28960  0 -> MPI_OK

[Thr 7336] MPI<23e90>18#4 GetOutbuf -1 f28960 65536 (0) -> 000000000FA289D0 20971520 MPI_OK

[Thr 7336] NiIGetServNo: servicename '56300' = port 56300

[Thr 7336] Address    Offset  Error response:

[Thr 7336] ------------------------------------------------------------------------

[Thr 7336] 000000000FA28A18  000000  48545450 2f312e30 20353030 204e6174 |HTTP/1.0 500 Nat|

[Thr 7336] 000000000FA28A28  000016  69766520 53534c20 6572726f 720d0a44 |ive SSL error..D|

[Thr 7336] 000000000FA28A38  000032  6174653a 20576564 2c203138 20466562 |ate: Wed, 18 Feb|

[Thr 7336] 000000000FA28A48  000048  20323031 35203133 3a32383a 32382047 | 2015 13:28:28 G|

[Thr 7336] 000000000FA28A58  000064  4d540d0a 53657276 65723a20 53415020 |MT..Server: SAP |

[Thr 7336] 000000000FA28A68  000080  4e657457 65617665 72204170 706c6963 |NetWeaver Applic|

[Thr 7336] 000000000FA28A78  000096  6174696f 6e205365 72766572 20372e32 |ation Server 7.2|

[Thr 7336] 000000000FA28A88  000112  31202f20 49434d20 372e3231 0d0a436f |1 / ICM 7.21..Co|

[Thr 7336] 000000000FA28A98  000128  6e6e6563 74696f6e 3a20636c 6f73650d |nnection: close.|

[Thr 7336] 000000000FA28AA8  000144  0a534150 2d49434d 434c4e54 4552524f |.SAP-ICMCLNTERRO|

[Thr 7336] 000000000FA28AB8  000160  523a2034 30370d0a 436f6e74 656e742d |R: 407..Content-|

[...]

[Thr 7336] MPI<23e90>18#5 FlushOutbuf -1 1 1 f28960 2206 6 -> 000000000FA289B0 MPI_OK

[Thr 7336] NiICloseHandle: shutdown and close hdl 373/sock 20200

[Thr 7336] IcmConnFreeContext: context 28 released

[Thr 7336] IcmServDecrRefCount: grpsapdx1.grouphc.net:56301 - new serv_ref_count: 0

[Thr 7336] IcmWorkerThread: Thread 1: Waiting for event

[Thr 4464] NiSelISelectInt: 0 handles selected (0 buffered)

[Thr 4464] NiIWrite: hdl 89 sent data (wrt=114,pac=1,MESG_IO)

[Thr 4464] MsINiWrite: sent 114 bytes

[Thr 4464] MsISnd2: send msg (ms hdr/msg 110/4 bytes) to name           MSG_SERVER, type 0, key -

[Thr 4464] MsSndName: MS_NOOP ok

[Thr 4464] Send 4 bytes to MSG_SERVER

[Thr 4464] MS_NOOP : asynchronous call

[Thr 4464] IcmMplxJncmKeepalive: keepalive sent to Message Server (keepalive 60 s)

[Thr 4464] NiIRead: hdl 89 received data (rcd=114,pac=1,MESG_IO)

[Thr 4464] NiSelISelectInt: 1 handles selected (1 buffered)

[Thr 4464] MsINiRead: received 114 bytes

[Thr 4464] MsIReceive: received msg (ms hdr/msg 110/4 bytes), flag 3, from MSG_SERVER          , typ 0, key -

[Thr 4464] JNCMIHdlNotification: handle opcode MS_NOOP(33)

[Thr 1332] Wed Feb 18 14:28:29 2015

[Thr 1332] SiSelNSelect: of 1 sockets 0 selected

As you can see,

we don´t have ANY message like "Chain of certificates is incomplete".

The problem is really strange, because it appers RANDOMLY:

it seems as if the receiver Gateway (UK Government Gateway) has some internal nodes (addressed by their Reverse Proxy / Load Balancer)

which fail to give back to SAP a proper response

(maybe because of expired certificates at internal nodes addressed by Reverse Proxy at UK GovernementGateway ??).

Any help/suggestion is really welcome!

Thank you

Gianluca

[Former Member · Accepted solution]

Hi,

I have also faced this kind of issue in the past(Some different integration) when the gateway fails to do a proper handshake with the servers and hence we get error message as ICM_HTTP_CONNECTION_FAILED.

I also collected the ICM traces with High trace settings.

I had searched a lot around that with close coordination with the gateway team but no response was there.

In my case we had automated jobs to process the error out messages from PI. No other solution...

Thanks.

[Former Member]

Hello Gianluca,

Even i have seen these intermittent connectivity errors with HMRC. Our scenarios were all synchronous so the only option here was to run the process again in SAP to send the request to HMRC. As mentioned by Gaurav dont see any other solution yet

Regards,

Sriram