Error establishing a mutual ssl between SMP server (java stack client) and SAP system.

[Former Member]

Hi,

I have problems establishing 2 way ssl connection between the SMP server (which acts as a client in this senario) which runs on java stack and SAP gateway system GKQ/G3T. I have imported the SAPnetCA.cer in SMP trustore and am able to establish a single(1-way ssl) between SMP and the gateway system. When i try establishing a 2-way ssl importing the signed CA into SMP keystore for G3T/GKQ i am not able to establish a connection.

the handshake logs for both server and client are provided below.

SAP GKQ server logs from SMICM

------------------------------------------------------------------------------------------------------------------------------

remote host: 10.21.84.209:57518 ()

    status: NOP

    connect time: 25.07.2013 14:30:44

    MPI request: <0>      MPI response: <0>

    request_buf_size: 0        response_buf_size:   0

    request_buf_used: 0        response_buf_used:   0

    request_buf_offset: 0        response_buf_offset: 0

[Thr 140389356902144] MPI 19: event flag already open:7959 key=16959

[Thr 140389356902144] MPI 19: event flag already open:7958 key=16958

[Thr 140389356902144] MPI:19 create pipe 7faef63e34d8 1

[Thr 140389356902144] MPI<115d>13#1 Open( ANONYMOUS 19 1 ) -> 19

[Thr 140389356902144] MPI<115d>13#2 Open( ANONYMOUS 19 0 ) -> 19

[Thr 140389356902144] MPI 1: event flag already open:7995 key=16995

[Thr 140389356902144] MPI 1: event flag already open:7994 key=16994

[Thr 140389356902144] MPI:1 create pipe 7faef63e13a8 1

[Thr 140389356902144] MPI<115e>1#1 Open( ANONYMOUS 1 0 ) -> 1

[Thr 140389356902144] MPI<115e>1#2 Open( ANONYMOUS 1 1 ) -> 1

[Thr 140389356902144] ->> SapSSLSessionInit(&sssl_hdl=7faef635b850, role=2 (SERVER), auth_type=1 (ASK_CLIENT_CERT))

[Thr 140389356902144] <<- SapSSLSessionInit()==SAP_O_K

[Thr 140389356902144]      in: args = "role=2 (SERVER), auth_type=1 (ASK_CLIENT_CERT)"

[Thr 140389356902144]     out: sssl_hdl = 12872f0

[Thr 140389356902144] ->> SapSSLSetNiHdl(sssl_hdl=12872f0, ni_hdl=384)

[Thr 140389356902144] NiIBlockMode: set blockmode for hdl 384 TRUE

[Thr 140389356902144]   SSL NI-sock: local=10.66.12.84:44380  peer=10.21.84.209:57518

[Thr 140389356902144] <<- SapSSLSetNiHdl(sssl_hdl=12872f0, ni_hdl=384)==SAP_O_K

[Thr 140389356902144] ->> SapSSLSessionStart(sssl_hdl=12872f0)

[Thr 140389356902144] NiIBlockMode: set blockmode for hdl 384 FALSE

[Thr 140389356902144] NiIHdlGetStatus: hdl 384/sock 50 ok, no data pending

[Thr 140389356902144] NiIBlockMode: set blockmode for hdl 384 TRUE

[Thr 140389356902144]   SSL_get_state() returned 0x000011a1 "SSLv3 read certificate verify B"

[Thr 140389356902144] *** ERROR during SecudeSSL_SessionStart() from SSL_accept()==SSL_ERROR_SSL

[Thr 140389356902144]    session uses PSE file "/usr/sap/GKQ/DVEBMGS80/sec/SAPSSLS.pse"

[Thr 140389356902144] SecudeSSL_SessionStart: SSL_accept() failed --

[Thr 140389356902144]   secude_error 536871698 (0x20000312) = "the client did not send a certificate verify handshake message for au

[Thr 140389356902144] >> ---------- Begin of Secude-SSL Errorstack ---------- >>

[Thr 140389356902144] ERROR in ssl3_get_cert_verify: (536871698/0x20000312) the client did not send a certificate verify handshake m

[Thr 140389356902144] << ---------- End of Secude-SSL Errorstack ----------

[Thr 140389356902144] <<- ERROR: SapSSLSessionStart(sssl_hdl=12872f0)==SSSLERR_SSL_ACCEPT

[Thr 140389356902144] ->> SapSSLErrorName(rc=-56)

[Thr 140389356902144] <<- SapSSLErrorName()==SSSLERR_SSL_ACCEPT

[Thr 140389356902144] *** ERROR => IcmConnInitServerSSL: SapSSLSessionStart returned (-56): SSSLERR_SSL_ACCEPT [icxxconn_mt. 1689]

[Thr 140389356902144] ->> SapSSLSessionDone(&sssl_hdl=7faef635b850)

[Thr 140389356902144] <<- SapSSLSessionDone()==SAP_O_K

[Thr 140389356902144]      in: sssl_hdl   = 12872f0

[Thr 140389356902144]          ... ni_hdl = 384

[Thr 140389356902144] NiICloseHandle: shutdown and close hdl 384/sock 50

[Thr 140389356902144] MPI<115d>13#3 Close( 13 ) opt=4 del=0( 1 0 ) wakeup=0-> MPI_OK

[Thr 140389356902144] MPI<115d>13#5 Delete( 13 ) -> MPI_OK

[Thr 140389356902144] MPI<115d>13#4 Close( 13 ) opt=4 del=1( 0 0 ) wakeup=0-> MPI_OK

[Thr 140389356902144] MPI<115e>1#3 Close( 1 ) opt=4 del=0( 0 1 ) wakeup=0-> MPI_OK

[Thr 140389356902144] MPI<115e>1#5 Delete( 1 ) -> MPI_OK

[Thr 140389356902144] MPI<115e>1#4 Close( 1 ) opt=4 del=1( 0 0 ) wakeup=0-> MPI_OK

-----------------------------------------------------------------------------------------------------------------------------------------------------------

SMP server(client) initializes the connection and verifies server cert and provides the client cert and then gets an unexpected error due to error on SAP gateway server

Java Client ssl handshake log:

-----------------------------------------------------------------------------------------------------------------------------------------------------------

*** Finished

verify_data:  { 15, 9, 188, 118, 38, 46, 235, 37, 204, 16, 76, 81 }

***

http-bio-8080-exec-7, WRITE: TLSv1 Handshake, length = 48

http-bio-8080-exec-7, READ: TLSv1 Alert, length = 2

http-bio-8080-exec-7, RECV TLSv1 ALERT:  fatal, unexpected_message

%% Invalidated:  [Session-9, TLS_RSA_WITH_AES_128_CBC_SHA]

http-bio-8080-exec-7, called closeSocket()

http-bio-8080-exec-7, handling exception: javax.net.ssl.SSLException: Received fatal alert: unexpected_message

http-bio-8080-exec-7, IOException in getSession():  javax.net.ssl.SSLException: Received fatal alert: unexpected_message

http-bio-8080-exec-7, called close()

http-bio-8080-exec-7, called closeInternal(true)

------------------------------------------------------------------------------------------------------------------------------------------------------------

the entire client log is in the attached file.

hope to get some help soon, Thanks in advance.

Regards,

Anil

[Ali_Chalhoub]

I found your question, let me know if that still an issue.