Re: [openssl-users] af_alg engine: digests work but ciphers do not?
I've spent the last few days configuring the cryptodev engine on my platform and I am also seeing the same behavior with that engine as I did with af_alg. It seems like OpenSSL does not even attempt to use the engine for ciphers. My next step is to start plowing through the openssl code to figure out how control is supposed to get passed to an engine - I believe this happens in evp_enc.c. Something tells me I'm just missing something fundamental here... like a configuration option on build or maybe something I missed in openssl.conf. Does anybody have any expertise in this? Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: openssl-users [openssl-users-boun...@openssl.org] on behalf of Barbe, Charles [charles.ba...@allworx.com] Sent: Wednesday, April 15, 2015 5:43 PM To: openssl-users@openssl.org Subject: [openssl-users] af_alg engine: digests work but ciphers do not? I'm wondering if anybody has any experience with the af_alg engine located here: http://src.carnivore.it/users/common/af_alg/about/ I am able to compile the engine and can run commands such as: openssl speed md5 -engine af_alg and I see that openssl has loaded the engine as indicated by this line in the output: engine af_alg set and can enable dmesg logging on the linux driver for my particular hardware and see that the driver is being used as expected. However, if I try to do a cipher instead of a digest, my driver is not used. For example, when I run: openssl speed aes-256-cbc -engine af_alg I see the engine get loaded but my dmesg logging indicates that the kernel driver was not used. And i get the same results for any of my supported ciphers. I have followed all the instructions in the URL located above including modifying my openssl.conf to include the proper configuration of the supported ciphers for my hardware but to no avail. Here is what the top of my openssl.conf looks like: # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME= . RANDFILE= $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids openssl_conf = openssl_def [openssl_def] engines = openssl_engines [openssl_engines] af_alg = af_alg_engine [af_alg_engine] default_algorithms = ALL CIPHERS=aes-128-cbc aes-192-cbc aes-256-cbc des-cbc des-ede3-cbc DIGESTS=md4 md5 sha1 sha224 sha256 sha512 # To use this configuration file with the -extfile option of the # openssl x509 utility, name here the section containing the # X.509v3 extensions to use: # extensions= # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca', 'req' and 'ts'. For reference, I am running the following: linux kernel v 3.19 openssl v 1.0.1m running on a TI am3352 Any help on why digests seem to be working with the af_alg engine but ciphers do not would be much appreciated. Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 -- This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments. ___ openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users ___ openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
Re: [openssl-users] af_alg engine: digests work but ciphers do not?
That was it... thank you so much!!! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: openssl-users [openssl-users-boun...@openssl.org] on behalf of Dr. Stephen Henson [st...@openssl.org] Sent: Tuesday, April 21, 2015 10:05 AM To: openssl-users@openssl.org Subject: Re: [openssl-users] af_alg engine: digests work but ciphers do not? On Tue, Apr 21, 2015, Barbe, Charles wrote: I've spent the last few days configuring the cryptodev engine on my platform and I am also seeing the same behavior with that engine as I did with af_alg. It seems like OpenSSL does not even attempt to use the engine for ciphers. My next step is to start plowing through the openssl code to figure out how control is supposed to get passed to an engine - I believe this happens in evp_enc.c. Something tells me I'm just missing something fundamental here... like a configuration option on build or maybe something I missed in openssl.conf. [snip] However, if I try to do a cipher instead of a digest, my driver is not used. For example, when I run: openssl speed aes-256-cbc -engine af_alg The speed command uses EVP for all digest operations but uses low level APIs for cipher operations when you don't include the -evp option. ENGINEs only work at the EVP level so your above command will just use the built in ciphers. If instead you do: openssl speed -evp aes-256-cbc It should work if you've set up an ENGINE to provide the default implementation. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org ___ openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users -- This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments. ___ openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
[openssl-users] af_alg engine: digests work but ciphers do not?
I'm wondering if anybody has any experience with the af_alg engine located here: http://src.carnivore.it/users/common/af_alg/about/ I am able to compile the engine and can run commands such as: openssl speed md5 -engine af_alg and I see that openssl has loaded the engine as indicated by this line in the output: engine af_alg set and can enable dmesg logging on the linux driver for my particular hardware and see that the driver is being used as expected. However, if I try to do a cipher instead of a digest, my driver is not used. For example, when I run: openssl speed aes-256-cbc -engine af_alg I see the engine get loaded but my dmesg logging indicates that the kernel driver was not used. And i get the same results for any of my supported ciphers. I have followed all the instructions in the URL located above including modifying my openssl.conf to include the proper configuration of the supported ciphers for my hardware but to no avail. Here is what the top of my openssl.conf looks like: # # OpenSSL example configuration file. # This is mostly being used for generation of certificate requests. # # This definition stops the following lines choking if HOME isn't # defined. HOME= . RANDFILE= $ENV::HOME/.rnd # Extra OBJECT IDENTIFIER info: #oid_file = $ENV::HOME/.oid oid_section = new_oids openssl_conf = openssl_def [openssl_def] engines = openssl_engines [openssl_engines] af_alg = af_alg_engine [af_alg_engine] default_algorithms = ALL CIPHERS=aes-128-cbc aes-192-cbc aes-256-cbc des-cbc des-ede3-cbc DIGESTS=md4 md5 sha1 sha224 sha256 sha512 # To use this configuration file with the -extfile option of the # openssl x509 utility, name here the section containing the # X.509v3 extensions to use: # extensions= # (Alternatively, use a configuration file that has only # X.509v3 extensions in its main [= default] section.) [ new_oids ] # We can add new OIDs in here for use by 'ca', 'req' and 'ts'. For reference, I am running the following: linux kernel v 3.19 openssl v 1.0.1m running on a TI am3352 Any help on why digests seem to be working with the af_alg engine but ciphers do not would be much appreciated. Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 -- This email message and any attachments are for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message and any attachments. ___ openssl-users mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
RE: Small memory leak on multithreaded server
Thank you all for your help. My problem did end up being my misunderstanding of the reference counting done by OpenSSL I was expecting that my X509*'s would get freed when I freed my SSL_CTX but that was an incorrect expectation because I still had a reference to those X509*'s in a vector. This conversation also helped me find some other places where I wasn't properly freeing reference counted OpenSSL structures. Thanks for the help! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Dr. Stephen Henson [st...@openssl.org] Sent: Friday, November 21, 2014 1:40 PM To: openssl-users@openssl.org Subject: Re: Small memory leak on multithreaded server On Fri, Nov 21, 2014, Barbe, Charles wrote: Yes... sorry, forgot to include this part of my shutdown sequence. One thing I am noticing is that I do not call X409_free on my certs. I even have a comment in my code saying that I am not freeing them because I think they will be freed when the SSL_CTX is freed. Is that a correct assumption or should I be calling X509 free on them explicitly? If you have an explicit X509 structure and you call SSL_CTX_use_certificate then the reference count is increased and you have to free up the certificate. However I'm a bit confused by the output. It *looks* like it is associated with a certificate verification operation which could be cached certificates in a store. Do you perform any operations with an X509_STORE structure? Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Small memory leak on multithreaded server
I see your point but I cannot have memory allocated when my application shuts down. This constraint is related to the fact that this is an embedded VoIP system. Thanks for the response! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Viktor Dukhovni [openssl-us...@dukhovni.org] Sent: Thursday, November 20, 2014 7:26 PM To: openssl-users@openssl.org Subject: Re: Small memory leak on multithreaded server On Thu, Nov 20, 2014 at 10:19:32PM +, Barbe, Charles wrote: I can do any combination of steps 2,3 and 4 above (ie. leave some of them out) and I always get the same amount of memory left over after I shut down my application. I believe this means that this is some sort of global information that OpenSSL is hanging on to and not something in my SSL connection structure. Specifically I get 20 blocks totaling 253 bytes. I have stack traces of where each block is allocated but I cannot figure out how this memory should be cleaned up. Each of the 20 blocks filter down to 1 of 5 root stack traces. The stack traces are: A fixed amount of memory that is not deallocated and is independent of the number of operations performed, is NOT a memory leak. Librariers to allocate memory for the lifetime of the process during one time initialization or first use of a function. This is normal. Tracking this down is a waste of time IMHO. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Small memory leak on multithreaded server
Thanks for the response... here is the code that runs when my connection closes:
void OpensslConnection::cleanup()
{
if(ssl != NULL)
{
if(isConnected)
{
while(SSL_shutdown(ssl) == 0)
;
}
SSL_free(ssl);
ERR_remove_state(0);
ssl = NULL;
}
isConnected = false;
}
And here is the code that runs to shut down my SSL library:
static void
openSslShutdown ()
{
CONF_modules_free();
ERR_remove_state(0);
CONF_modules_unload(1);
ERR_free_strings();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
if (opensslLocks != NULL)
{
for(int i = 0; i CRYPTO_num_locks(); i++)
{
PAL_mutexDestroy (opensslLocks[i]);
}
IST_FREE (opensslLocks);
}
}
Also, I have numerous worker threads handling connections and they all do the
following before they exit:
ERR_remove_thread_state(0);
Charles A. Barbe
Senior Software Engineer
Allworx, a Windstream company
245 East Main St | Rochester NY | 14604
charles.ba...@allworx.com | 585.421.5565
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on
behalf of Jeffrey Walton [noloa...@gmail.com]
Sent: Thursday, November 20, 2014 6:03 PM
To: OpenSSL Users List
Subject: Re: Small memory leak on multithreaded server
Any help would be appreciated.
This could be one of two problems. First, it could be an issue with
your code and the way you handle cleanup. To help diagnose this,
please show us your cleanup code.
Second, it could be the memory leak from the compression methods. This
is a well known problem dating back a few years that has not been
fixed. See, for example,
http://rt.openssl.org/Ticket/Display.html?id=2561user=guestpass=guest.
On Thu, Nov 20, 2014 at 5:19 PM, Barbe, Charles
charles.ba...@allworx.com wrote:
Hello,
I have noticed a small and consistent memory leak in my multithreaded openssl
server and am wondering if somebody can help me figure out what I need to do
to free it when my application closes. I am on OpenSSL version 1.0.1j. Here's
how I reproduce the leak:
1) Start up my server
2) Load my homepage with Chrome
3) Load my homepage with IE
4) Load my homepage with Firefox
I can do any combination of steps 2,3 and 4 above (ie. leave some of them
out) and I always get the same amount of memory left over after I shut down
my application. I believe this means that this is some sort of global
information that OpenSSL is hanging on to and not something in my SSL
connection structure.
Specifically I get 20 blocks totaling 253 bytes. I have stack traces of where
each block is allocated but I cannot figure out how this memory should be
cleaned up. Each of the 20 blocks filter down to 1 of 5 root stack traces.
The stack traces are:
Repeated 6 times:
0x085c519c: asn1_item_ex_combine_new at tasn_new.c:164
0x085c7fd8: ASN1_item_ex_d2i at tasn_dec.c:315 (discriminator 1)
0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706
0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616
0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195
0x085c80a8: ASN1_item_d2i at tasn_dec.c:136
0x0867b368: X509V3_EXT_d2i at v3_lib.c:177
0x08682f3c: x509v3_cache_extensions at v3_purp.c:464
0x08683698: x509v3_cache_extensions at v3_purp.c:722
(inlined by) X509_check_issued at v3_purp.c:706
0x0866fddc: check_issued at x509_vfy.c:417
0x086716d0: X509_verify_cert at x509_vfy.c:212
0x086967c0: ssl3_output_cert_chain at s3_both.c:387
0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365
0x0868b864: ssl3_accept at s3_srvr.c:418
0x086982f4: ssl23_get_client_hello at s23_srvr.c:657
0x08698bb8: ssl23_accept at s23_srvr.c:213
Repeated 6 times:
0x085d12a4: ASN1_STRING_set at asn1_lib.c:389
0x085c715c: asn1_ex_c2i at tasn_dec.c:1056
0x085c7334: asn1_d2i_ex_primitive at tasn_dec.c:836
0x085c7fc8: ASN1_item_ex_d2i at tasn_dec.c:198
0x085c82a0: asn1_template_noexp_d2i at tasn_dec.c:734
0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616
0x085c7a20: ASN1_item_ex_d2i at tasn_dec.c:331
0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706
0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616
0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195
0x085c80a8: ASN1_item_d2i at tasn_dec.c:136
0x0867b368: X509V3_EXT_d2i at v3_lib.c:177
0x08682f3c: x509v3_cache_extensions at v3_purp.c:464
0x08683698: x509v3_cache_extensions at v3_purp.c:722
(inlined by) X509_check_issued at v3_purp.c:706
0x0866fddc: check_issued at x509_vfy.c:417
0x086716d0: X509_verify_cert at x509_vfy.c:212
0x086967c0: ssl3_output_cert_chain at s3_both.c:387
0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365
0x0868b864: ssl3_accept at s3_srvr.c:418
0x086982f4: ssl23_get_client_hello at s23_srvr.c:657
0x08698bb8: ssl23_accept at s23_srvr.c:213
Repeated 6 times:
0x085d13bc: ASN1_STRING_type_new at asn1_lib.c:429
0x085c71f4
RE: Small memory leak on multithreaded server
It sort of makes sense that this is a stack of certs that is left over but what confuses me is that this memory is allocated by calls executed as a result of my calling SSL_accept(ssl). Also, we're talking 253 bytes... i would imagine that stack of certs would be larger than that, wouldn't it? And shouldn't all the memory allocated by SSL_accept be freed when I call SSL_shutdown and SSL_free? What looks suspicious to me is the calls to x509v3_cache_extensions that are in the traces below. This implies to me that openssl is caching something. How do I ask it to clear that cache? If i need to augment the library to do so, where are the pointers to the objects it is caching so I can free it? I tried to trace the code but got lost... i'm sure I can trace it given enough time but thought I'd ask the community first. I also note that I have no indication of any left over memory from my allocation of the certs that I am providing to my SSL_CTX... meaning that I believe I am cleaning my SSL_CTX and associated resources appropriately. The memory in question is definitely allocated by OpenSSL as a result of calling SSL_accept... I just need to figure out how to ask OpenSSL to free it. Thanks so much for the help. Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Dr. Stephen Henson [st...@openssl.org] Sent: Thursday, November 20, 2014 9:50 PM To: openssl-users@openssl.org Subject: Re: Small memory leak on multithreaded server On Thu, Nov 20, 2014, Barbe, Charles wrote: Hello, I have noticed a small and consistent memory leak in my multithreaded openssl server and am wondering if somebody can help me figure out what I need to do to free it when my application closes. I am on OpenSSL version 1.0.1j. Here's how I reproduce the leak: 1) Start up my server 2) Load my homepage with Chrome 3) Load my homepage with IE 4) Load my homepage with Firefox I can do any combination of steps 2,3 and 4 above (ie. leave some of them out) and I always get the same amount of memory left over after I shut down my application. I believe this means that this is some sort of global information that OpenSSL is hanging on to and not something in my SSL connection structure. Specifically I get 20 blocks totaling 253 bytes. I have stack traces of where each block is allocated but I cannot figure out how this memory should be cleaned up. Each of the 20 blocks filter down to 1 of 5 root stack traces. The stack traces are: Repeated 6 times: 0x085c519c: asn1_item_ex_combine_new at tasn_new.c:164 0x085c7fd8: ASN1_item_ex_d2i at tasn_dec.c:315 (discriminator 1) 0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Repeated 6 times: 0x085d12a4: ASN1_STRING_set at asn1_lib.c:389 0x085c715c: asn1_ex_c2i at tasn_dec.c:1056 0x085c7334: asn1_d2i_ex_primitive at tasn_dec.c:836 0x085c7fc8: ASN1_item_ex_d2i at tasn_dec.c:198 0x085c82a0: asn1_template_noexp_d2i at tasn_dec.c:734 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7a20: ASN1_item_ex_d2i at tasn_dec.c:331 0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Repeated 6 times: 0x085d13bc: ASN1_STRING_type_new at asn1_lib.c:429 0x085c71f4: asn1_ex_c2i at tasn_dec.c:1032 0x085c7334: asn1_d2i_ex_primitive at tasn_dec.c:836 0x085c7fc8: ASN1_item_ex_d2i at tasn_dec.c:198 0x085c82a0: asn1_template_noexp_d2i at tasn_dec.c:734 0x085c85a0
RE: Small memory leak on multithreaded server
Yes... sorry, forgot to include this part of my shutdown sequence. One thing I
am noticing is that I do not call X409_free on my certs. I even have a comment
in my code saying that I am not freeing them because I think they will be freed
when the SSL_CTX is freed. Is that a correct assumption or should I be calling
X509 free on them explicitly?
Here's the extra code that I forgot to include:
STATUS
SSL_shutDown()
{
// Client
if (clientCtxt != NULL)
{
SSL_CTX_free (clientCtxt);
clientCtxt = NULL;
}
// View
clearCertsVector (getCertCachePointer (SSL_TYPE_VIEW));
if (pViewKeyCache != NULL)
{
EVP_PKEY_free (pViewKeyCache);
pViewKeyCache = NULL;
}
if (viewServerCtxt != NULL)
{
SSL_CTX_free (viewServerCtxt);
viewServerCtxt = NULL;
}
// Web - only if server supports HTTPS
if (supportHTTPS)
{
clearCertsVector (getCertCachePointer (SSL_TYPE_WEB));
if (pPendingWebKeyCache != NULL)
{
EVP_PKEY_free (pPendingWebKeyCache);
pPendingWebKeyCache = NULL;
}
if (pWebKeyCache != NULL)
{
EVP_PKEY_free (pWebKeyCache);
pWebKeyCache = NULL;
}
if (webServerCtxt != NULL)
{
SSL_CTX_free (webServerCtxt);
webServerCtxt = NULL;
}
}
CHECK_911 (BIO_free (out) 0);
openSslShutdown ();
return SUCCESS;
}
static void
clearCertsVector(CERTS_VECTOR* certs)
{
//Not freeing certs because they will be
// freed when context is destroyed
if(certs != NULL)
{
certs-clear();
}
}
Charles A. Barbe
Senior Software Engineer
Allworx, a Windstream company
245 East Main St | Rochester NY | 14604
charles.ba...@allworx.com | 585.421.5565
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on
behalf of Dr. Stephen Henson [st...@openssl.org]
Sent: Friday, November 21, 2014 12:26 PM
To: openssl-users@openssl.org
Subject: Re: Small memory leak on multithreaded server
On Fri, Nov 21, 2014, Barbe, Charles wrote:
Thanks for the response... here is the code that runs when my connection
closes:
void OpensslConnection::cleanup()
{
if(ssl != NULL)
{
if(isConnected)
{
while(SSL_shutdown(ssl) == 0)
;
}
SSL_free(ssl);
ERR_remove_state(0);
ssl = NULL;
}
isConnected = false;
}
And here is the code that runs to shut down my SSL library:
static void
openSslShutdown ()
{
CONF_modules_free();
ERR_remove_state(0);
CONF_modules_unload(1);
ERR_free_strings();
EVP_cleanup();
CRYPTO_cleanup_all_ex_data();
if (opensslLocks != NULL)
{
for(int i = 0; i CRYPTO_num_locks(); i++)
{
PAL_mutexDestroy (opensslLocks[i]);
}
IST_FREE (opensslLocks);
}
}
Also, I have numerous worker threads handling connections and they all do the
following before they exit:
ERR_remove_thread_state(0);
You are calling SSL_CTX_free aren't you?
Steve.
--
Dr Stephen N. Henson. OpenSSL project core developer.
Commercial tech support now available see: http://www.openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
RE: Small memory leak on multithreaded server
I did a search through my application code and see no occurrence of the string X509_STORE so I don't believe I'm doing any operations with that type of structure. I am using SSL_CTX_use_certificate and was possibly under the incorrect assumption that my code did not have to explicitly free the X509* that I pass to that argument if I am subsequently calling SSL_CTX_free on the CTX. In retrospect that doesn't sound correct. I will fix my code to free the X509s as well as the CTX and see if that is my issue. Thank you everybody for all of the help! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Dr. Stephen Henson [st...@openssl.org] Sent: Friday, November 21, 2014 1:40 PM To: openssl-users@openssl.org Subject: Re: Small memory leak on multithreaded server On Fri, Nov 21, 2014, Barbe, Charles wrote: Yes... sorry, forgot to include this part of my shutdown sequence. One thing I am noticing is that I do not call X409_free on my certs. I even have a comment in my code saying that I am not freeing them because I think they will be freed when the SSL_CTX is freed. Is that a correct assumption or should I be calling X509 free on them explicitly? If you have an explicit X509 structure and you call SSL_CTX_use_certificate then the reference count is increased and you have to free up the certificate. However I'm a bit confused by the output. It *looks* like it is associated with a certificate verification operation which could be cached certificates in a store. Do you perform any operations with an X509_STORE structure? Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Small memory leak on multithreaded server
Hello, I have noticed a small and consistent memory leak in my multithreaded openssl server and am wondering if somebody can help me figure out what I need to do to free it when my application closes. I am on OpenSSL version 1.0.1j. Here's how I reproduce the leak: 1) Start up my server 2) Load my homepage with Chrome 3) Load my homepage with IE 4) Load my homepage with Firefox I can do any combination of steps 2,3 and 4 above (ie. leave some of them out) and I always get the same amount of memory left over after I shut down my application. I believe this means that this is some sort of global information that OpenSSL is hanging on to and not something in my SSL connection structure. Specifically I get 20 blocks totaling 253 bytes. I have stack traces of where each block is allocated but I cannot figure out how this memory should be cleaned up. Each of the 20 blocks filter down to 1 of 5 root stack traces. The stack traces are: Repeated 6 times: 0x085c519c: asn1_item_ex_combine_new at tasn_new.c:164 0x085c7fd8: ASN1_item_ex_d2i at tasn_dec.c:315 (discriminator 1) 0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Repeated 6 times: 0x085d12a4: ASN1_STRING_set at asn1_lib.c:389 0x085c715c: asn1_ex_c2i at tasn_dec.c:1056 0x085c7334: asn1_d2i_ex_primitive at tasn_dec.c:836 0x085c7fc8: ASN1_item_ex_d2i at tasn_dec.c:198 0x085c82a0: asn1_template_noexp_d2i at tasn_dec.c:734 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7a20: ASN1_item_ex_d2i at tasn_dec.c:331 0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Repeated 6 times: 0x085d13bc: ASN1_STRING_type_new at asn1_lib.c:429 0x085c71f4: asn1_ex_c2i at tasn_dec.c:1032 0x085c7334: asn1_d2i_ex_primitive at tasn_dec.c:836 0x085c7fc8: ASN1_item_ex_d2i at tasn_dec.c:198 0x085c82a0: asn1_template_noexp_d2i at tasn_dec.c:734 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7a20: ASN1_item_ex_d2i at tasn_dec.c:331 0x085c81dc: asn1_template_noexp_d2i at tasn_dec.c:706 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Once: 0x0866380c: sk_new at stack.c:125 0x0866380c: sk_new at stack.c:125 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at tasn_dec.c:136 0x0867b368: X509V3_EXT_d2i at v3_lib.c:177 0x08682f3c: x509v3_cache_extensions at v3_purp.c:464 0x08683698: x509v3_cache_extensions at v3_purp.c:722 (inlined by) X509_check_issued at v3_purp.c:706 0x0866fddc: check_issued at x509_vfy.c:417 0x086716d0: X509_verify_cert at x509_vfy.c:212 0x086967c0: ssl3_output_cert_chain at s3_both.c:387 0x0868ac68: ssl3_send_server_certificate at s3_srvr.c:3365 0x0868b864: ssl3_accept at s3_srvr.c:418 0x086982f4: ssl23_get_client_hello at s23_srvr.c:657 0x08698bb8: ssl23_accept at s23_srvr.c:213 Once: 0x08663990: sk_insert at stack.c:151 0x085c8204: asn1_template_noexp_d2i at tasn_dec.c:715 0x085c85a0: asn1_template_ex_d2i at tasn_dec.c:616 0x085c7d90: ASN1_item_ex_d2i at tasn_dec.c:195 0x085c80a8: ASN1_item_d2i at
RE: Certificate problem - SOLVED
I figured it out and am now wondering if there is a defect in the openssl verify command. This suggestion from Dave Thompson: I would first try x509 -noout -subject|issuer -nameopt multiline,show_type and see if that helps. Pointed me in the right direction. What i found was that Issuer for certificate A, which was the one that was NOT working, looked like this: [cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt multiline,show_type -in CertA.pem issuer= countryName = UTF8STRING:US stateOrProvinceName = UTF8STRING:New York organizationName = UTF8STRING:Allworx Corp, a Windstream Company commonName= UTF8STRING:view While the issuer for certificate B and subject for my CA looked like this: [cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt multiline,show_type -in CertB.pem issuer= countryName = PRINTABLESTRING:US stateOrProvinceName = UTF8STRING:New York organizationName = UTF8STRING:Allworx Corp, a Windstream Company commonName= UTF8STRING:view [cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt multiline,show_type -in CA.pem issuer= countryName = PRINTABLESTRING:US stateOrProvinceName = UTF8STRING:New York organizationName = UTF8STRING:Allworx Corp, a Windstream Company commonName= UTF8STRING:view So it looks like openssl verify is not taking the type of countryName into account while the browsers are. Is this expected behavior or a defect? Again, thanks for all the help everybody! CHAD From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Dave Thompson [dthomp...@prinpay.com] Sent: Monday, July 07, 2014 4:03 PM To: openssl-users@openssl.org Subject: RE: Certificate problem From: owner-openssl-us...@openssl.org On Behalf Of Barbe, Charles Sent: Sunday, July 06, 2014 22:42 I have the following certificates and associated private keys: A - certificate A generated with one version of my software not using openssl B - certificate B generated with a new version of my software that does use openssl CA - a local certificate authority whose private key is used to sign both A and B I can verify both A and B using openssl verify using CA as the cafile argument. However, when I install CA on a client and try to connect a web browser to my server running the two different versions of software, they complain that they cannot find the issuer with A but not with B. I have examined both certificates and cannot find anything different about them. As far as I can tell, the only difference is that B used openssl to generate the certificate and A used our own custom software. The odd thing to me is that openssl verify can verify both just fine. What are the web browsers doing different? I've tried chrome, Firefox and opera and all behave the same... Accepting B and rejecting A. Does anybody have any suggestions on where to look to figure this out? A tool to use? You are installing in the correct placeS which can be different per browser, right? The only thing that springs to mind that could be invisible is string types and some options of the cert Issuer fields vs the CA Subject. RFC 5280 requires a fairly complicated Unicode-aware comparison algorithm which I believe openssl does (it definitely canonicalizes before comparison, but I haven't gone through the canonicalization to make sure it exactly matches the RFC); browsers might not do the same (perhaps indirectly) although I'd be surprised if NONE do. I would first try x509 -noout -subject|issuer -nameopt multiline,show_type and see if that helps. __ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org CA.pem Description: CA.pem CertA.pem Description: CertA.pem
RE: Certificate problem - SOLVED
Yet openssl verify said OK to both of my certificates against the CA
certificate... so is it incorrectly neglecting to compare the types when it
tries to build the chain of certificates?
Charles A. Barbe
Senior Software Engineer
Allworx, a Windstream company
245 East Main St | Rochester NY | 14604
charles.ba...@allworx.com | 585.421.5565
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on
behalf of Jeffrey Walton [noloa...@gmail.com]
Sent: Tuesday, July 08, 2014 4:19 PM
To: OpenSSL Users List
Subject: Re: Certificate problem - SOLVED
On Tue, Jul 8, 2014 at 3:39 PM, Barbe, Charles
charles.ba...@allworx.com wrote:
I figured it out and am now wondering if there is a defect in the openssl
verify command. This suggestion from Dave Thompson:
I would first try x509 -noout -subject|issuer -nameopt multiline,show_type
and see if that helps.
Pointed me in the right direction. What i found was that Issuer for
certificate A, which was the one that was NOT working, looked like this:
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CertA.pem
issuer=
countryName = UTF8STRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
While the issuer for certificate B and subject for my CA looked like this:
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CertB.pem
issuer=
countryName = PRINTABLESTRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CA.pem
issuer=
countryName = PRINTABLESTRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
So it looks like openssl verify is not taking the type of countryName into
account while the browsers are. Is this expected behavior or a defect?
Not sure if this is any consolation, but countryName is a
DirectoryString, and PrintableString is OK per RFC 5280
(http://tools.ietf.org/html/rfc5280#section-4.1.2.6):
DirectoryString ::= CHOICE {
teletexString TeletexString (SIZE (1..MAX)),
printableString PrintableString (SIZE (1..MAX)),
universalString UniversalString (SIZE (1..MAX)),
utf8String UTF8String (SIZE (1..MAX)),
bmpString BMPString (SIZE (1..MAX)) }
However, there is the following on page 23:
When encoding attribute values of type DirectoryString, conforming
CAs MUST use PrintableString or UTF8String encoding, with the
following exceptions:
(a) When the subject of the certificate is a CA, the subject
field MUST be encoded in the same way as it is encoded in the
issuer field (Section 4.1.2.4) in all certificates issued by
the subject CA. Thus, if the subject CA encodes attributes
in the issuer fields of certificates that it issues using the
TeletexString, BMPString, or UniversalString encodings, then
the subject field of certificates issued to that CA MUST use
the same encoding.
So the DirectoryString must be the same type. You can't make it
utf8String in the server certificate's issuer and PrintableString in
the CA's subject.
Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
RE: Certificate problem - SOLVED
Also don't these lines of the spec:
countryName ATTRIBUTE ::= {
WITH SYNTAX PrintableString (SIZE (2))
-- IS 3166 codes only
ID id-at-countryName }
indicate that countryName is restricted to the PrintableString (SIZE (2)) type
which is a restricted subset of DirectoryString?
Charles A. Barbe
Senior Software Engineer
Allworx, a Windstream company
245 East Main St | Rochester NY | 14604
charles.ba...@allworx.com | 585.421.5565
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on
behalf of Barbe, Charles [charles.ba...@allworx.com]
Sent: Tuesday, July 08, 2014 4:44 PM
To: openssl-users@openssl.org
Subject: RE: Certificate problem - SOLVED
Yet openssl verify said OK to both of my certificates against the CA
certificate... so is it incorrectly neglecting to compare the types when it
tries to build the chain of certificates?
Charles A. Barbe
Senior Software Engineer
Allworx, a Windstream company
245 East Main St | Rochester NY | 14604
charles.ba...@allworx.com | 585.421.5565
From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on
behalf of Jeffrey Walton [noloa...@gmail.com]
Sent: Tuesday, July 08, 2014 4:19 PM
To: OpenSSL Users List
Subject: Re: Certificate problem - SOLVED
On Tue, Jul 8, 2014 at 3:39 PM, Barbe, Charles
charles.ba...@allworx.com wrote:
I figured it out and am now wondering if there is a defect in the openssl
verify command. This suggestion from Dave Thompson:
I would first try x509 -noout -subject|issuer -nameopt multiline,show_type
and see if that helps.
Pointed me in the right direction. What i found was that Issuer for
certificate A, which was the one that was NOT working, looked like this:
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CertA.pem
issuer=
countryName = UTF8STRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
While the issuer for certificate B and subject for my CA looked like this:
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CertB.pem
issuer=
countryName = PRINTABLESTRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
[cbarbe@localhost foropensslusers]$ openssl x509 -noout -issuer -nameopt
multiline,show_type -in CA.pem
issuer=
countryName = PRINTABLESTRING:US
stateOrProvinceName = UTF8STRING:New York
organizationName = UTF8STRING:Allworx Corp, a Windstream Company
commonName= UTF8STRING:view
So it looks like openssl verify is not taking the type of countryName into
account while the browsers are. Is this expected behavior or a defect?
Not sure if this is any consolation, but countryName is a
DirectoryString, and PrintableString is OK per RFC 5280
(http://tools.ietf.org/html/rfc5280#section-4.1.2.6):
DirectoryString ::= CHOICE {
teletexString TeletexString (SIZE (1..MAX)),
printableString PrintableString (SIZE (1..MAX)),
universalString UniversalString (SIZE (1..MAX)),
utf8String UTF8String (SIZE (1..MAX)),
bmpString BMPString (SIZE (1..MAX)) }
However, there is the following on page 23:
When encoding attribute values of type DirectoryString, conforming
CAs MUST use PrintableString or UTF8String encoding, with the
following exceptions:
(a) When the subject of the certificate is a CA, the subject
field MUST be encoded in the same way as it is encoded in the
issuer field (Section 4.1.2.4) in all certificates issued by
the subject CA. Thus, if the subject CA encodes attributes
in the issuer fields of certificates that it issues using the
TeletexString, BMPString, or UniversalString encodings, then
the subject field of certificates issued to that CA MUST use
the same encoding.
So the DirectoryString must be the same type. You can't make it
utf8String in the server certificate's issuer and PrintableString in
the CA's subject.
Jeff
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
__
OpenSSL Project http
Re: Certificate problem
I am positive that I am installing the ca in the correct spot because connections to server B correctly show the CA cert as the trusted root when I view the certificate for the connection in the web browser. To be clear, openssl verify says that both certificates A and B are ok when I provide my ca certificate as the CAfile argument. It is web browsers that do not find the issuer for certificate A. Ie, it seems to me like the browsers are MORE stringent in their checks than openssl verify. I will ask my management if I can attach the certs tomorrow... You and I know there is no risk attaching them but i still need the approval. Thanks for responding! CHAD On Jul 7, 2014, at 4:03 PM, Dave Thompson dthomp...@prinpay.com wrote: From: owner-openssl-us...@openssl.org On Behalf Of Barbe, Charles Sent: Sunday, July 06, 2014 22:42 I have the following certificates and associated private keys: A - certificate A generated with one version of my software not using openssl B - certificate B generated with a new version of my software that does use openssl CA - a local certificate authority whose private key is used to sign both A and B I can verify both A and B using openssl verify using CA as the cafile argument. However, when I install CA on a client and try to connect a web browser to my server running the two different versions of software, they complain that they cannot find the issuer with A but not with B. I have examined both certificates and cannot find anything different about them. As far as I can tell, the only difference is that B used openssl to generate the certificate and A used our own custom software. The odd thing to me is that openssl verify can verify both just fine. What are the web browsers doing different? I've tried chrome, Firefox and opera and all behave the same... Accepting B and rejecting A. Does anybody have any suggestions on where to look to figure this out? A tool to use? You are installing in the correct placeS which can be different per browser, right? The only thing that springs to mind that could be invisible is string types and some options of the cert Issuer fields vs the CA Subject. RFC 5280 requires a fairly complicated Unicode-aware comparison algorithm which I believe openssl does (it definitely canonicalizes before comparison, but I haven't gone through the canonicalization to make sure it exactly matches the RFC); browsers might not do the same (perhaps indirectly) although I'd be surprised if NONE do. I would first try x509 -noout -subject|issuer -nameopt multiline,show_type and see if that helps. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Certificate problem
As I said in another note, I will try to send the certs tomorrow. Thanks for the help! CHAD On Jul 7, 2014, at 4:42 PM, Kyle Hamilton aerow...@gmail.com wrote: On 7/6/2014 7:41 PM, Barbe, Charles wrote: Does anybody have any suggestions on where to look to figure this out? A tool to use? I realize that actually attaching the certa might be helpful but I do not have them handy as I write this. Please let me know if that might help somebody help me figure this out. Attaching the certs would be most helpful. Or, opening a ticket that includes them. Being unable to find the issuer for B but not A suggests that it's the Issuer field, or the IssuerKeyIdentifier; however, it's not guaranteed. (If only X.509 weren't so obtuse, if only PKIX weren't so tied to it, but they are.) -Kyle H __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Certificate problem
I will try an ASN.1 decoder tomorrow. Thanks for the suggestion! One thing I did try today was to have both servers generate their certificates using the same private key. Theoretically I would expect the two certs to then be exactly the same to the bit... I am not providing any domain or ip specific fields just so that I can do this comparison and made sure all other variable fields would be static. The only variable left should be my signing algorithm vs the one used my openssl's code. What I think I found was that the two certs were identical except for 4 bytes. There was a 0x05 and 0x00 following two fields in the open ssl generated cert. Each occurrence of these 2 bytes was following the signature algorithm identifier (in two places I think). These 4 bytes were not in the non-open ssl cert. could this be my problem? Is there a significance to the 0x05 and 0x00? They seemed to be part of the enclosing structure that contained the signature alg id but not part of the id itself. At least according to wireshark. Are they necessary padding that I'm missing in my custom cert generation? Like I said earlier, I'll try to attach the certs tomorrow. I really appreciate everybody's help! CHAD On Jul 7, 2014, at 5:09 PM, Ben Wilson b...@digicert.com wrote: You could try examining both PEM-encoded certificates using an ASN.1 decoder, such as the one here - http://lapo.it/asn1js -Original Message- From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org] On Behalf Of Barbe, Charles Sent: Sunday, July 6, 2014 8:42 PM To: openssl-users@openssl.org Subject: Certificate problem I'm having a problem with generating certificates and I'm wondering if anybody has any suggestions on where to look. I have the following certificates and associated private keys: A - certificate A generated with one version of my software not using openssl B - certificate B generated with a new version of my software that does use openssl CA - a local certificate authority whose private key is used to sign both A and B I can verify both A and B using openssl verify using CA as the cafile argument. However, when I install CA on a client and try to connect a web browser to my server running the two different versions of software, they complain that they cannot find the issuer with A but not with B. I have examined both certificates and cannot find anything different about them. As far as I can tell, the only difference is that B used openssl to generate the certificate and A used our own custom software. The odd thing to me is that openssl verify can verify both just fine. What are the web browsers doing different? I've tried chrome, Firefox and opera and all behave the same... Accepting B and rejecting A. Does anybody have any suggestions on where to look to figure this out? A tool to use? I realize that actually attaching the certa might be helpful but I do not have them handy as I write this. Please let me know if that might help somebody help me figure this out. Thanks! Chad. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Certificate problem
CHAD On Jul 7, 2014, at 11:11 PM, Jeffrey Walton noloa...@gmail.com wrote: On Mon, Jul 7, 2014 at 9:59 PM, Barbe, Charles charles.ba...@allworx.com wrote: I will try an ASN.1 decoder tomorrow. Thanks for the suggestion! One thing I did try today was to have both servers generate their certificates using the same private key. Theoretically I would expect the two certs to then be exactly the same to the bit... I am not providing any domain or ip specific fields just so that I can do this comparison and made sure all other variable fields would be static. The serial numbers will likely be different. I explicitly set the serial to 1 so I could do a binary compare of the certs. I'm beginning to think it could be my protocol implementation and not the cert itself. Jeff __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Re: Certificate problem
CHAD On Jul 7, 2014, at 11:03 PM, Dave Thompson dthomp...@prinpay.com wrote: From: owner-openssl-us...@openssl.org On Behalf Of Barbe, Charles Sent: Monday, July 07, 2014 21:59 I will try an ASN.1 decoder tomorrow. Thanks for the suggestion! One thing I did try today was to have both servers generate their certificates using the same private key. Theoretically I would expect the two certs to then be exactly the same to the bit... I am not providing any domain or ip specific fields just so that I can do this comparison and made sure all other variable fields would be static. The only variable left should be my signing If these certs are (intended to be) for the same server(s), then the server identity (usually name, rarely IP) can be the same, but it should not be omitted. SSL clients are supposed to, and at least browsers do, enforce it. Yep I understand this. The removal of the name / ip was just for testing so I could do a binary compare of the two certs. Every cert should have a serial# which should be unique, and for real CA certs normally isn't even serial, it's random. OpenSSL still normally does serial. Did you do something to fake the serial, or did you ignore that (small) difference? In our case we're doing random but, again for my test only, I explicitly set it to 1. algorithm vs the one used my openssl's code. What I think I found was that the two certs were identical except for 4 bytes. There was a 0x05 and 0x00 following two fields in the open ssl generated cert. Each occurrence of these 2 bytes was following the signature algorithm identifier (in two places I think). These 4 bytes were not in the non-open ssl cert. could this be my problem? Is there a significance to the 0x05 and 0x00? They seemed to be part of the enclosing structure that contained the signature alg id but not part of the id itself. At least according to wireshark. Are they necessary padding that I'm missing in my custom cert generation? They aren't necessary. Yes, the AlgorithmIdentifier occurs in two places; X.509 was designed in early days when there was great concern over the possibility of algorithm substitution attacks on publickey crypto as there historically had been on some symmetric crypto, so it occurs in the signed content (near the beginning) as well as after. The AlgorithmIdentifier is a general structure used in numerous places for numerous purposes. In some of these uses it includes 'parameters' which basically specialize the algorithm. In this use, the parameters are not needed, and ASN.1 allows two ways of handling this: the parameters can be omitted entirely, or they can be encoded as an ASN.1 NULL, which is the bytes 05 00. A robust parser/verifier should accept both. This makes me think that my problem may be in our implementation of the protocol and not in the certificate itself. Thanks for the help! Like I said earlier, I'll try to attach the certs tomorrow. I really appreciate everybody's help! FYI ASN.1 decode can also be done by openssl commandline 'asn1parse', not as flexibly as some but it's already right to hand. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Certificate problem
I'm having a problem with generating certificates and I'm wondering if anybody has any suggestions on where to look. I have the following certificates and associated private keys: A - certificate A generated with one version of my software not using openssl B - certificate B generated with a new version of my software that does use openssl CA - a local certificate authority whose private key is used to sign both A and B I can verify both A and B using openssl verify using CA as the cafile argument. However, when I install CA on a client and try to connect a web browser to my server running the two different versions of software, they complain that they cannot find the issuer with A but not with B. I have examined both certificates and cannot find anything different about them. As far as I can tell, the only difference is that B used openssl to generate the certificate and A used our own custom software. The odd thing to me is that openssl verify can verify both just fine. What are the web browsers doing different? I've tried chrome, Firefox and opera and all behave the same... Accepting B and rejecting A. Does anybody have any suggestions on where to look to figure this out? A tool to use? I realize that actually attaching the certa might be helpful but I do not have them handy as I write this. Please let me know if that might help somebody help me figure this out. Thanks! Chad. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
RE: Using single EVP_PKEY instance across multiple connections
To expand on this question a little more, is it safe to just create one SSL_CTX* at initialization of my server that will be used each time a new client connects when i do SSL_new(ctx)? Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 From: owner-openssl-us...@openssl.org [owner-openssl-us...@openssl.org] on behalf of Barbe, Charles [charles.ba...@allworx.com] Sent: Friday, June 27, 2014 4:18 PM To: openssl-users@openssl.org Subject: Using single EVP_PKEY instance across multiple connections Hi, I have an implementation of a multi-threaded HTTP server that I wrote using OpenSSL version 1.0.1g. Currently, on initialization of the server, I load my private key from disk and store it in an EVP_PKEY pointer. Whenever I accept a new connection, I use that same pointer to an EVP_PKEY in my call to SSL_CTX_use_PrivateKey. Is that safe or should I be copying my EVP_PKEY for each connection? Thanks! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Using single EVP_PKEY instance across multiple connections
Hi, I have an implementation of a multi-threaded HTTP server that I wrote using OpenSSL version 1.0.1g. Currently, on initialization of the server, I load my private key from disk and store it in an EVP_PKEY pointer. Whenever I accept a new connection, I use that same pointer to an EVP_PKEY in my call to SSL_CTX_use_PrivateKey. Is that safe or should I be copying my EVP_PKEY for each connection? Thanks! Charles A. Barbe Senior Software Engineer Allworx, a Windstream company 245 East Main St | Rochester NY | 14604 charles.ba...@allworx.com | 585.421.5565 __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
Conflicting information regarding non-blocking sockets
Hello, I have implemented an SSL server on the ecos platform and it seems to be working well however I am a tiny bit unclear about the requirements around whether or not calls to SSL_read and SSL_write need to be syncrhonized and I want to avoid doing something incorrect. The specific question I have is this: If I call SSL_read() and openSSL tells me that it NEEDS_WRITE or NEEDS_READ, do I need to then hold off calls to SSL_write until I have called SSL_read again and it is succeeds? And then same question but with SSL_write. I am asking because I have seen conflicting information. Here is where I'm getting the conflicting information: From the calls must be synchronized camp: In chapter 5, pages 159-166 of Network Security with OpenSSL by John Viega, Matt Messier Pravir Chandra, 2002 they state very clearly that once we've made a call to an SSL I/O function, and it requires a retry, we should not call other I/O functions until the original call has succeeded. It even gives a code example that achieves this synchronization. From the calls to SSL_read and SSL_write can be treated independently camp: In this thread from the openssl-dev mailing list: http://marc.info/?l=openssl-devm=124308191325560w=2 A gentleman named David Schwartz states, I'm not sure where you see that, but that is obviously bogus. Suppose we have a protocol that permits either side to read or write at any time. We call SSL_read in case the other side sent something, it returns WANT_READ, because the other side didn't send anything. Are you seriously arguing that we now can't send anything until the other side does? Isn't that obviously an impossible requirement? In response to this statement by somebody else Because the documentation appears in several places to state that when I receive WANT_X from _any_ OpenSSL API operation in non-blocking mode, the next API operation I issue on that SSL * must be the exact same API operation, with the same parameters. Can somebody please tell me which of these two sources are correct or if I am mis-reading them? Thanks! CHAD __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager majord...@openssl.org
