RE: memory growing when using SSL connections
I had a similar issue that was fixed by moving the CRYPTO_cleanup_all_ex_data
command to the end of the application (as has been suggested). Have you tried
running Valgrind with the application? This may tell you where the memory leak
is occurring.
--Kevin
p.s. You will need to compile OpenSSL with the -DPURIFY option (and you may
need to grab a recent patch that I mailed to openssl-dev to get -DPURIFY
working properly).
--Kevin
-Original Message-
From: owner-openssl-us...@openssl.org [mailto:owner-openssl-us...@openssl.org]
On Behalf Of David
Sent: Monday, January 11, 2010 11:48 AM
To: openssl-users@openssl.org
Subject: Re: memory growing when using SSL connections
Hi Jeremy,
I did try removing the CRYPTO_cleanup_all_ex_data() call, based on Dr.
Henson's response, but I still have the same problem.
I bypassed the SSL calls and used pure telnet to my server and there
were no signs of the application growing.
Incidentally, I don't see this problem when running on a Solaris box.
Perhaps, it has something to do with the AIX environment.
Regards,
David
--
From: "Jeremy Hunt"
Sent: Sunday, January 10, 2010 6:56 PM
To:
Subject: Re: memory growing when using SSL connections
> Hi David et al,
>
> On reading the responses so far two new thoughts occur to me:
>
> 1. In view of Dr Henson's response, I wonder if removing the
> CRYPTO_cleanup_all_ex_data() call in your loop will fix the problem.
> Perhaps reusing the context structure after calling it may have the
> reverse effect.
>
> 2. It may not be an SSL problem at all. Can you remove the SSL calls from
> your application and see if you still get the memory leak? Your underlying
> telnet application may be the cause.
>
> Good Luck,
>
> Jeremy
>
> Dr. Stephen Henson wrote:
>> [safeTgram (safetgram-in) receive status: NOT encrypted, NOT signed.]
>>
>>
>> On Thu, Jan 07, 2010, David wrote:
>>
>>
>>> Hi,
>>>
>>> I'm using tn3270 sessions running over SSL. I may have up to 124
>>> sessions activated concurrently, although I plan to get up to 250
>>> sessions at some point.
>>> Whenever the sessions are stopped and restarted, I notice intermittently
>>> that memory grows in multiples of 4K bytes.
>>> I'm running on AIX 5.1, 5.2 and 5.3 and using openssl-0.9.8l. There
>>> doesn't appear to be an obvious memory leak in either my application or
>>> the OpenSSL stuff (all memory allocated when the sessions are started
>>> are freed when the sessions are stopped).
>>> Here's a summary of the code structure:
>>>
>>> SSL_library_init();
>>> meth = TLSv1_client_method();
>>> RAND_seed();
>>> ctx = SSL_CTX_new(meth);
>>>
>>> while ([some telnet connection wants to do SSL])
>>> {
>>> ssl = SSL_new(ctx);
>>> SSL_set_fd()
>>> SSL_set_cipher_list(); SSL_set_connect_state();
>>> SSL_connect();
>>> do SSL_read(), SSL_write()
>>> SSL_shutdown();
>>> close FD;
>>> SSL_free();
>>> CRYPTO_cleanup_all_ex_data();
>>> }
>>> Any ideas would be appreciated. Thanks,
>>> David
>>>
>>
>> Some cleanups occur on each connection and others only when the
>> application
>> shuts down.
>>
>> You should *not* call CRYPTO_cleanup_all_ex_data() on every SSL
>> connection
>> because later SSL connections may use it and end up not freeing data
>> correctly.
>>
>> This is especially an issue if connections use compression (OpenSSL
>> compiled
>> against zlib) as it is by default in some linux distributions.
>>
>> Steve.
>> --
>> Dr Stephen N. Henson. OpenSSL project core developer.
>> Commercial tech support now available see: http://www.openssl.org
>> __
>> OpenSSL Project http://www.openssl.org
>> User Support Mailing Listopenssl-users@openssl.org
>> Automated List Manager majord...@openssl.org
>>
>>
>
>
> --
>
> "The most likely way for the world to be destroyed, most experts agree, is
> by accident. That's where we come in; we're computer professionals. We
> cause accidents." -- Nathaniel Borenstein, co-creator of MIME
> __
> OpenSSL Project http://www.openssl.org
> User Support Mailing Listopenssl-users@openssl.org
> Automated List Manager majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager majord...@openssl.org
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automate
Running in production with -DPURIFY?
It is reasonable/Ok to run a production system built with -DPURIFY? Is that extra bit of entropy critical? Sincerely, Kevin Regan F5 Networks
Calling SSL_CTX_load_verify_locations multiple times?
I have an application that calls "SSL_CTX_load_verify_locations" on a single file that contains multiple CA certificates. It then calls "SSL_load_client_CA_file" on this file to extract the CA subject names. Finally, it calls "SSL_CTX_set_client_CA_list" to set the list of CA subject names that are sent to the client. This all works fine. However, I would like to add additional code, further on in the process that calls "SSL_CTX_load_verify_locations" again for other certificates to add those certificates to the CA certificates in the SSL context (not overwrite the current certificates). In addition, I would like to call "SSL_CTX_add_client_CA" for each one of these new certificates to add them to the list of CA subject names returned to the client. After adding this additional code, I am seeing the following error: SSL error:140890B2:SSL routines:SSL3_GET_CLIENT_CERTIFICATE:no certificate returned Is what I am trying to do supported? If so, is there something I'm doing wrong? If not, is there another way to accomplish this (adding CA certificates from multiple files into a single SSL context)? Thanks, Kevin Kevin Regan Principal Software Engineer F5 Networks
RE: Possible memory leak with RSA_private_encrypt?
After recompiling with debugging information, I'm getting a bit more
information:
==21592== 24 bytes in 1 blocks are definitely lost in loss record 1 of 1
==21592==at 0x4421846: malloc (vg_replace_malloc.c:149)
==21592==by 0x4464D40: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44653F6: CRYPTO_malloc (in
/usr/lib/libcrypto.so.0.9.8)
==21592==by 0x4495FB9: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x449628A: bn_expand2 (in /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x4496ACA: BN_bin2bn (in /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44B9F63: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44B8631: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44BB0EA: RSA_private_encrypt (in
/usr/lib/libcrypto.so.0.9.8)
==21592==by 0x80490E7: main (in /shared/rsa_test1)
--Kevin
-Original Message-
From: Kevin Regan
Sent: Sunday, January 13, 2008 10:26 AM
To: openssl-users@openssl.org
Subject: RE: Possible memory leak with RSA_private_encrypt?
I've also tested a version with the following change:
...
//
// Encrypt test data.
//
for (int i = 0; i < ; i++) {
...
}
//
// Free RSA key.
//
...
This causes the application to continually grow in memory size. So, the
memory definitely seems to be leaked in the RSA_private_encrypt call.
--Kevin
-Original Message-
From: [EMAIL PROTECTED] on behalf of Kevin Regan
Sent: Sun 1/13/2008 12:30 AM
To: openssl-users@openssl.org
Subject: Possible memory leak with RSA_private_encrypt?
There is a good chance that I am doing something wrong. However, the
program below produces the following output from valgrind with OpenSSL
v0.9.8g:
==12647== 1,032 bytes in 43 blocks are definitely lost in loss record 2
of 2
==12647==at 0x4421846: malloc (vg_replace_malloc.c:149)
==12647==by 0x4464FFD: default_malloc_ex (in
/usr/lib/libcrypto.so.0.9.8)
Here is the program (replace keyFile with valid path to key):
#include
#include
#include
#include
#include
#include
#include
#include
#include
namespace {
std::string keyFile = "/my/pem/key";
const size_t PKCS1_PADDING_LENGTH = 11; }
int
main(int argc, char *argv[])
{
//
// Open key file.
//
std::FILE* file = std::fopen(keyFile.c_str(), "r");
if (NULL == file) {
std::cerr << "Could not open key file" << std::endl;
std::exit(1);
}
//
// Read in private key.
//
RSA* privKey = PEM_read_RSAPrivateKey(file, NULL, NULL, (void*)"");
std::fclose(file);
if (NULL == privKey) {
std::cerr << "Could not read private key" << std::endl;
std::exit(1);
}
//
// Create test data.
//
std::vector data;
for (int i = 0; i < 5000; i++) {
data.push_back(i % 256);
}
//
// Encrypt test data.
//
int outputBlockSize = RSA_size(privKey);
int inputBlockSize = outputBlockSize - PKCS1_PADDING_LENGTH;
std::vector encryptedData;
std::vector tempEncryptedData(outputBlockSize);
for (size_t i = 0; i < data.size(); i += inputBlockSize) {
int bytesLeft = data.size() - i;
int bytesToEncrypt =
bytesLeft > inputBlockSize ? inputBlockSize : bytesLeft;
int result = RSA_private_encrypt(bytesToEncrypt,
&(data[i]),
&(tempEncryptedData[0]),
privKey,
RSA_PKCS1_PADDING);
if (result != outputBlockSize) {
std::cerr << "Could not RSA encrypt authentication token."
<< std::endl;
std::exit(1);
}
encryptedData.insert(encryptedData.end(),
tempEncryptedData.begin(),
tempEncryptedData.end());
}
//
// Free RSA key.
//
RSA_free(privKey);
CONF_modules_unload(1);
EVP_cleanup();
ENGINE_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_remove_state(0);
ERR_free_strings();
std::cout << "Data encrypted" << std::endl;
return 0;
} // main
<>
RE: Possible memory leak with RSA_private_encrypt?
I found the issue. There is a custom patch used locally that seems to
be leaking memory. I'm going to find the responsible coworker and make
them pay for wasting out time. ;-)
--Kevin
-Original Message-
From: Kevin Regan
Sent: Monday, January 14, 2008 10:59 AM
To: 'openssl-users@openssl.org'
Subject: RE: Possible memory leak with RSA_private_encrypt?
After recompiling with debugging information, I'm getting a bit more
information:
==21592== 24 bytes in 1 blocks are definitely lost in loss record 1 of 1
==21592==at 0x4421846: malloc (vg_replace_malloc.c:149)
==21592==by 0x4464D40: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44653F6: CRYPTO_malloc (in
/usr/lib/libcrypto.so.0.9.8)
==21592==by 0x4495FB9: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x449628A: bn_expand2 (in /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x4496ACA: BN_bin2bn (in /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44B9F63: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44B8631: (within /usr/lib/libcrypto.so.0.9.8)
==21592==by 0x44BB0EA: RSA_private_encrypt (in
/usr/lib/libcrypto.so.0.9.8)
==21592==by 0x80490E7: main (in /shared/rsa_test1)
--Kevin
-Original Message-----
From: Kevin Regan
Sent: Sunday, January 13, 2008 10:26 AM
To: openssl-users@openssl.org
Subject: RE: Possible memory leak with RSA_private_encrypt?
I've also tested a version with the following change:
...
//
// Encrypt test data.
//
for (int i = 0; i < ; i++) {
...
}
//
// Free RSA key.
//
...
This causes the application to continually grow in memory size. So, the
memory definitely seems to be leaked in the RSA_private_encrypt call.
--Kevin
-Original Message-
From: [EMAIL PROTECTED] on behalf of Kevin Regan
Sent: Sun 1/13/2008 12:30 AM
To: openssl-users@openssl.org
Subject: Possible memory leak with RSA_private_encrypt?
There is a good chance that I am doing something wrong. However, the
program below produces the following output from valgrind with OpenSSL
v0.9.8g:
==12647== 1,032 bytes in 43 blocks are definitely lost in loss record 2
of 2
==12647==at 0x4421846: malloc (vg_replace_malloc.c:149)
==12647==by 0x4464FFD: default_malloc_ex (in
/usr/lib/libcrypto.so.0.9.8)
Here is the program (replace keyFile with valid path to key):
#include
#include
#include
#include
#include
#include
#include
#include
#include
namespace {
std::string keyFile = "/my/pem/key";
const size_t PKCS1_PADDING_LENGTH = 11; }
int
main(int argc, char *argv[])
{
//
// Open key file.
//
std::FILE* file = std::fopen(keyFile.c_str(), "r");
if (NULL == file) {
std::cerr << "Could not open key file" << std::endl;
std::exit(1);
}
//
// Read in private key.
//
RSA* privKey = PEM_read_RSAPrivateKey(file, NULL, NULL, (void*)"");
std::fclose(file);
if (NULL == privKey) {
std::cerr << "Could not read private key" << std::endl;
std::exit(1);
}
//
// Create test data.
//
std::vector data;
for (int i = 0; i < 5000; i++) {
data.push_back(i % 256);
}
//
// Encrypt test data.
//
int outputBlockSize = RSA_size(privKey);
int inputBlockSize = outputBlockSize - PKCS1_PADDING_LENGTH;
std::vector encryptedData;
std::vector tempEncryptedData(outputBlockSize);
for (size_t i = 0; i < data.size(); i += inputBlockSize) {
int bytesLeft = data.size() - i;
int bytesToEncrypt =
bytesLeft > inputBlockSize ? inputBlockSize : bytesLeft;
int result = RSA_private_encrypt(bytesToEncrypt,
&(data[i]),
&(tempEncryptedData[0]),
privKey,
RSA_PKCS1_PADDING);
if (result != outputBlockSize) {
std::cerr << "Could not RSA encrypt authentication token."
<< std::endl;
std::exit(1);
}
encryptedData.insert(encryptedData.end(),
tempEncryptedData.begin(),
tempEncryptedData.end());
}
//
// Free RSA key.
//
RSA_free(privKey);
CONF_modules_unload(1);
EVP_cleanup();
ENGINE_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_remove_state(0);
ERR_free_strings();
std::cout << "Data encrypted" << std::endl;
return 0;
} // main
<>
RE: Possible memory leak with RSA_private_encrypt?
I've also tested a version with the following change:
...
//
// Encrypt test data.
//
for (int i = 0; i < ; i++) {
...
}
//
// Free RSA key.
//
...
This causes the application to continually grow in memory size. So, the memory
definitely seems to be leaked in the RSA_private_encrypt call.
--Kevin
-Original Message-
From: [EMAIL PROTECTED] on behalf of Kevin Regan
Sent: Sun 1/13/2008 12:30 AM
To: openssl-users@openssl.org
Subject: Possible memory leak with RSA_private_encrypt?
There is a good chance that I am doing something wrong. However, the program
below produces the following output from valgrind with OpenSSL v0.9.8g:
==12647== 1,032 bytes in 43 blocks are definitely lost in loss record 2 of 2
==12647==at 0x4421846: malloc (vg_replace_malloc.c:149)
==12647==by 0x4464FFD: default_malloc_ex (in /usr/lib/libcrypto.so.0.9.8)
Here is the program (replace keyFile with valid path to key):
#include
#include
#include
#include
#include
#include
#include
#include
#include
namespace {
std::string keyFile = "/my/pem/key";
const size_t PKCS1_PADDING_LENGTH = 11;
}
int
main(int argc, char *argv[])
{
//
// Open key file.
//
std::FILE* file = std::fopen(keyFile.c_str(), "r");
if (NULL == file) {
std::cerr << "Could not open key file" << std::endl;
std::exit(1);
}
//
// Read in private key.
//
RSA* privKey = PEM_read_RSAPrivateKey(file, NULL, NULL, (void*)"");
std::fclose(file);
if (NULL == privKey) {
std::cerr << "Could not read private key" << std::endl;
std::exit(1);
}
//
// Create test data.
//
std::vector data;
for (int i = 0; i < 5000; i++) {
data.push_back(i % 256);
}
//
// Encrypt test data.
//
int outputBlockSize = RSA_size(privKey);
int inputBlockSize = outputBlockSize - PKCS1_PADDING_LENGTH;
std::vector encryptedData;
std::vector tempEncryptedData(outputBlockSize);
for (size_t i = 0; i < data.size(); i += inputBlockSize) {
int bytesLeft = data.size() - i;
int bytesToEncrypt =
bytesLeft > inputBlockSize ? inputBlockSize : bytesLeft;
int result = RSA_private_encrypt(bytesToEncrypt,
&(data[i]),
&(tempEncryptedData[0]),
privKey,
RSA_PKCS1_PADDING);
if (result != outputBlockSize) {
std::cerr << "Could not RSA encrypt authentication token."
<< std::endl;
std::exit(1);
}
encryptedData.insert(encryptedData.end(),
tempEncryptedData.begin(),
tempEncryptedData.end());
}
//
// Free RSA key.
//
RSA_free(privKey);
CONF_modules_unload(1);
EVP_cleanup();
ENGINE_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_remove_state(0);
ERR_free_strings();
std::cout << "Data encrypted" << std::endl;
return 0;
} // main
<>
Possible memory leak with RSA_private_encrypt?
There is a good chance that I am doing something wrong. However, the program
below produces the following output from valgrind with OpenSSL v0.9.8g:
==12647== 1,032 bytes in 43 blocks are definitely lost in loss record 2 of 2
==12647==at 0x4421846: malloc (vg_replace_malloc.c:149)
==12647==by 0x4464FFD: default_malloc_ex (in /usr/lib/libcrypto.so.0.9.8)
Here is the program (replace keyFile with valid path to key):
#include
#include
#include
#include
#include
#include
#include
#include
#include
namespace {
std::string keyFile = "/my/pem/key";
const size_t PKCS1_PADDING_LENGTH = 11;
}
int
main(int argc, char *argv[])
{
//
// Open key file.
//
std::FILE* file = std::fopen(keyFile.c_str(), "r");
if (NULL == file) {
std::cerr << "Could not open key file" << std::endl;
std::exit(1);
}
//
// Read in private key.
//
RSA* privKey = PEM_read_RSAPrivateKey(file, NULL, NULL, (void*)"");
std::fclose(file);
if (NULL == privKey) {
std::cerr << "Could not read private key" << std::endl;
std::exit(1);
}
//
// Create test data.
//
std::vector data;
for (int i = 0; i < 5000; i++) {
data.push_back(i % 256);
}
//
// Encrypt test data.
//
int outputBlockSize = RSA_size(privKey);
int inputBlockSize = outputBlockSize - PKCS1_PADDING_LENGTH;
std::vector encryptedData;
std::vector tempEncryptedData(outputBlockSize);
for (size_t i = 0; i < data.size(); i += inputBlockSize) {
int bytesLeft = data.size() - i;
int bytesToEncrypt =
bytesLeft > inputBlockSize ? inputBlockSize : bytesLeft;
int result = RSA_private_encrypt(bytesToEncrypt,
&(data[i]),
&(tempEncryptedData[0]),
privKey,
RSA_PKCS1_PADDING);
if (result != outputBlockSize) {
std::cerr << "Could not RSA encrypt authentication token."
<< std::endl;
std::exit(1);
}
encryptedData.insert(encryptedData.end(),
tempEncryptedData.begin(),
tempEncryptedData.end());
}
//
// Free RSA key.
//
RSA_free(privKey);
CONF_modules_unload(1);
EVP_cleanup();
ENGINE_cleanup();
CRYPTO_cleanup_all_ex_data();
ERR_remove_state(0);
ERR_free_strings();
std::cout << "Data encrypted" << std::endl;
return 0;
} // main
Problem with client certificate authentication.
I get the following error on the client: 24611:error:1409441B:SSL routines:SSL3_READ_BYTES:tlsv1 alert decrypt error:s3_pkt.c:985:SSL alert number 51 24611:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:226: and on the server: 24610:error:0407006A:rsa routines:RSA_padding_check_PKCS1_type_1:block type is not 01:rsa_pk1.c:100: 24610:error:04067072:rsa routines:RSA_EAY_PUBLIC_DECRYPT:padding check failed:rsa_eay.c:459: 24610:error:1408807A:SSL routines:SSL3_GET_CERT_VERIFY:bad rsa signature:s3_srvr.c:1635: 24610:error:140780E5:SSL routines:SSL23_READ:ssl handshake failure:s23_lib.c:180: When attempting to do client authentication (with SSL_CTX_set_verify on the server). I've created the certificate and key programatically using the OpenSSL API. The client seems to have no problem verifying the server certificate, but the server dies when trying to verify the client. Any ideas? Sincerely, Kevin Regan Kevin Regan Technical Lead Houston UNIX Team Office: 2200 Phone: 713-548-1767 __ OpenSSL Project http://www.openssl.org User Support Mailing List[EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
