Hello Matt, The SSL_get_error() returns 5(SSL_ERROR_SYSCALL) It does not print anything for this error, just an empty string. I use the following to print error but nothing is printed if ((retVal = SSL_accept(m_pSsl)) < 1) { sslError = SSL_get_error(m_pSsl, retVal); LOGERROR(getOpenSSLError()); throw dwRet;// eSSL_ERROR; } string getOpenSSLError() { BIO *bio = BIO_new(BIO_s_mem()); ERR_print_errors(bio); char *buf; size_t len = BIO_get_mem_data(bio, &buf); string ret(buf, len); BIO_free(bio); return ret; }
*Kamala Ayyar* 502 Claremont Ave. Teaneck NJ 07666-2563 Tel: (201)530-0861 On Fri, Feb 4, 2022 at 10:54 AM Matt Caswell <m...@openssl.org> wrote: > > > On 04/02/2022 15:17, Kamala Ayyar wrote: > > > > Hello, > > > > We are facing a strange handshake failure issue with a test server and > > client application using OpenSSL in Windows. We have tried with both > > 1.1.1g and 3.0.1 versions- same problem. We created a Dll to handle the > > OpenSSL functions- where the SSL context, SSL object and certificates > > are handled. The certificates are obtained from the Windows store and > > converted to cert and key using PKCS12_parse() > > The server accepts non secure connection from the client and then passes > > the socket to the Dll that calls the TLS_server_method() and creates the > > SSL context, SSL object and loads the certificates for use. It however > > fails at SSL_accept(m_pSsl). We use a call > > back SSL_set_info_callback(m_pSsl, apps_ssl_info_callback) that gave us > > the following error information > > SSL_accept:Error in before SSL initialization > > On the client side the same Dll is called with a client > > method TLS_client_method() and the error displayed is SSL_connect:Error > > in SSLv3/TLS write client hello > > We have confirmed the certificates are good and valid. > > > > The same Dll called from a different heavily threaded application with > > over 2000+ clients works well and handshake connections established > > without issues on a different port number. > > > > We have also tried to use OpenSSL methods directly without using the Dll > > but we get the same failure. This was also used with server and client > > on the same machine as well as different machines with the same > > outcome. The non secure communication works fine between the server and > > the client > > What does SSL_get_error() report after SSL_accept() fails? > > Also please dump the OpenSSL error stack when it fails, e.g. using > something like ERR_print_errors_fp(stdout); > > Matt >