I have an asynchronous win32 websocket server (written in C/C++ using MSVS
2010) application that I now want to support WSS - a WebSocket Secure
connection. To accomplish this, I added openssl to my application.
However, when the client tries to connect to my webserver, openssl is
rejecting the client during the initial handshake with the following error:
13500:error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown
protocol:.\ssl\s23_clnt.c:683:
Here is some other info to help in uncovering the cause for this error:
* openssl version - OpenSSL 1.0.0d 8 Feb 2011
* I am testing locally using the latest Firefox/chrome browsers.
IP=127.0.0.1 Port=8181
* I created my own certificates for testing
Create the root CA
>openssl req -newkey rsa:1024 -sha1 -keyout rootkey.pem -out
rootreq.pem
>openssl x509 -req -in rootreq.pem -sha1 -extensions v3_ca -signkey
rootkey.pem -out rootcert.pem
>cat rootcert.pem rootkey.pem > root.pem
Create the server's certificate and sign it with the root CA
>openssl req -newkey rsa:1024 -sha1 -keyout serverkey.pem -out
serverreq.pem
>openssl x509 -req -in serverreq.pem -sha1 -extensions usr_cert -CA
root.pem -CAkey root.pem -CAcreateserial -out servercert.pem
>cat servercert.pem serverkey.pem rootcert.pem > server.pem
Create the client certificate and sign it with the root CA
>openssl req -newkey rsa:1024 -sha1 -keyout clientkey.pem -out
clientreq.pem
>openssl x509 -req -in clientreq.pem -sha1 -extensions usr_cert -CA
root.pem -CAkey root.pem -CAcreateserial -out clientcert.pem
>cat clientcert.pem clientkey.pem rootcert.pem > client.pem
Create the dh512.pem dh1024.pem
>openssl dhparam -check -text -5 512 -out dh512.pem
>openssl dhparam -check -text -5 1024 -out dh1024.pem
* I setup the SSL_CTX for the server as follows:
SSL_CTX *setup_server_ctx (void)
{
// Performs the necessary actions to provide certificate data
// to the client.
//
SSL_CTX *ctx;
// Specify the SSL protocol for the server to use.
ctx = SSL_CTX_new (SSLv23_method ());
if (ctx == NULL) {
int_error("Error creating a new SSL_CTX object");
return (NULL);
}
// Load the trusted certificates from rootcert.pem
if (SSL_CTX_load_verify_locations(ctx, CAFILE, CADIR) != 1) {
int_error("Error loading CA file and/or directory");
SSL_CTX_free (ctx);
return (NULL);
}
// Load the built-in certificate stores.
if (SSL_CTX_set_default_verify_paths(ctx) != 1) {
int_error("Error loading default CA file and/or directory");
SSL_CTX_free (ctx);
return (NULL);
}
// Incorporate certificate information into the SSL_CTX by
loading
// a chain of certificates from the specified file name
if (SSL_CTX_use_certificate_chain_file (ctx, SERVER_CERTFILE)
!= 1) {
int_error ("Error loading certificate from file");
SSL_CTX_free (ctx);
return (NULL);
}
// Set password required to decrypt an encrypted private key
SSL_CTX_set_default_passwd_cb(ctx, passwd_cb);
// Load in the application's private key
if (SSL_CTX_use_PrivateKey_file (ctx, SERVER_CERTFILE,
SSL_FILETYPE_PEM) != 1) {
int_error ("Error loading private key from file");
SSL_CTX_free (ctx);
return (NULL);
}
// Have the server request a certificate fom the client.
// or if no certificate supplied.
SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, verify_callback);
// Set the maximum depth for verification.
SSL_CTX_set_verify_depth(ctx, 4);
// Enable all bug workarounds, disable SSL version 2, and
recompute the private
// part of the DH exchange for each client connecting.
SSL_CTX_set_options(ctx, SSL_OP_ALL | SSL_OP_NO_SSLv2 |
SSL_OP_SINGLE_DH_USE);
SSL_CTX_set_tmp_dh_callback(ctx, tmp_dh_callback);
if (SSL_CTX_set_cipher_list(ctx, CIPHER_LIST) != 1) {
int_error("Error setting cipher list (no valid ciphers)");
SSL_CTX_free (ctx);
return (NULL);
}
return (ctx);
} // setup_server_ctx
* Since I am using a asynchronous winsock implementation (FD_READ,
FD_WRITE, etc) to handle TCP packets, I am using memory bios to process the
data. I based some of my code off of the following implementation:
http://funcptr.net/2012/04/08/openssl-as-a-filter-%28or-non-blocking-openssl%29/
Can anyone help me figure out what is happening? Any help would be
appreciated, especially since I am new to working with OpenSSL. Thanks.
