Re: Investigating a leak
On 7/19/05, Rich Salz [EMAIL PROTECTED] wrote:
Doing this now, but since my app is a server, it never really reaches
the end, and the leaks build up during its lifetime, which I'd rather
hoped could be more than a day or so. As it is now, the process'
virtual memory size quickly reaches 3.5 GB, and then malloc() starts
failing.
Something wrong is happening. Lots of folks are running apache/openssl
servers for days, weeks, etc., at a time.
If you're seeing memory leaks like you describe, then it is far more
likely that the like is either in your code, or there's a bug in the way
you're calling OpenSSL. Do you get the same growth when openssl isn't
used? Make sure you xxx_free every object you xxx_new.
/r$
Hi,
I was looking at the code ... could it be that the memory leak is in
the main server loop?
Sometimes, the ssl_accept may return 0, but with an error code like
SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, which just means that
could not complete the handshake, but not really an error (just call
ssl_accept again ... ).
But in you main server thread, you just do a continue, thus looping
again ... and creating a brand new ssl structure and socket ... you
should
loop() {
create the tcp socket
loop () {
ssl_accept from the tcp socket till non-zero
}
}
Take a look at the examples that come with openssl.
Regards,
Cesc
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: Investigating a leak
That'd do it. But if you're doing the loop in that sequence, and if you have
set non-blocking on the sockets, then instead of polling for the connection
you can use select or poll (depending on your platform) to wait for incoming
connection activity, then enter ssl_accept in blocking mode which would
probably cut down on those fake errors. Wouldn't this be a better sequence?
If you are worried about inserting another call in the sequence... Being in
blocking mode at this stage shouldn't adversely affect performance as the
connection thread would only be in a polling loop during this period anyway
in this code example. Being in a time-limited wait during select only aids
performance as it allows the client serving threads access to the time-slice
for the process which would otherwise be tied up in the polling loop. The
new select() call could adversely impact performance if you have to service
a continual stream of new client connections that prevent the select() from
blocking at all, but that is unlikely in the real world.
Regards,
Simon
- --
Simon Edwards
-Original Message-
From: Cesc [mailto:[EMAIL PROTECTED]
Sent: 19 July 2005 09:58
To: openssl-users@openssl.org
Subject: Re: Investigating a leak
On 7/19/05, Rich Salz [EMAIL PROTECTED] wrote:
Doing this now, but since my app is a server, it never really reaches
the end, and the leaks build up during its lifetime, which I'd rather
hoped could be more than a day or so. As it is now, the process'
virtual memory size quickly reaches 3.5 GB, and then malloc() starts
failing.
Something wrong is happening. Lots of folks are running apache/openssl
servers for days, weeks, etc., at a time.
If you're seeing memory leaks like you describe, then it is far more
likely that the like is either in your code, or there's a bug in the way
you're calling OpenSSL. Do you get the same growth when openssl isn't
used? Make sure you xxx_free every object you xxx_new.
/r$
Hi,
I was looking at the code ... could it be that the memory leak is in
the main server loop?
Sometimes, the ssl_accept may return 0, but with an error code like
SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, which just means that
could not complete the handshake, but not really an error (just call
ssl_accept again ... ).
But in you main server thread, you just do a continue, thus looping
again ... and creating a brand new ssl structure and socket ... you
should
loop() {
create the tcp socket
loop () {
ssl_accept from the tcp socket till non-zero
}
}
Take a look at the examples that come with openssl.
Regards,
Cesc
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
This message has been scanned for viruses by MailController -
www.MailController.altohiway.com
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: Investigating a leak
Why don't you move the ssl handshake to the thread code?
Let the main loop accept tcp sockets, then create the thread with the
tcp socket as the argument you pass to the thread run method ...
perform then the ssl_accept loop (blocking or non-blocking, up to you)
in the thread ...
Just a suggestion.
Cesc
On 7/19/05, Simon Edwards [EMAIL PROTECTED] wrote:
That'd do it. But if you're doing the loop in that sequence, and if you have
set non-blocking on the sockets, then instead of polling for the connection
you can use select or poll (depending on your platform) to wait for incoming
connection activity, then enter ssl_accept in blocking mode which would
probably cut down on those fake errors. Wouldn't this be a better sequence?
If you are worried about inserting another call in the sequence... Being in
blocking mode at this stage shouldn't adversely affect performance as the
connection thread would only be in a polling loop during this period anyway
in this code example. Being in a time-limited wait during select only aids
performance as it allows the client serving threads access to the time-slice
for the process which would otherwise be tied up in the polling loop. The
new select() call could adversely impact performance if you have to service
a continual stream of new client connections that prevent the select() from
blocking at all, but that is unlikely in the real world.
Regards,
Simon
- --
Simon Edwards
-Original Message-
From: Cesc [mailto:[EMAIL PROTECTED]
Sent: 19 July 2005 09:58
To: openssl-users@openssl.org
Subject: Re: Investigating a leak
On 7/19/05, Rich Salz [EMAIL PROTECTED] wrote:
Doing this now, but since my app is a server, it never really reaches
the end, and the leaks build up during its lifetime, which I'd rather
hoped could be more than a day or so. As it is now, the process'
virtual memory size quickly reaches 3.5 GB, and then malloc() starts
failing.
Something wrong is happening. Lots of folks are running apache/openssl
servers for days, weeks, etc., at a time.
If you're seeing memory leaks like you describe, then it is far more
likely that the like is either in your code, or there's a bug in the way
you're calling OpenSSL. Do you get the same growth when openssl isn't
used? Make sure you xxx_free every object you xxx_new.
/r$
Hi,
I was looking at the code ... could it be that the memory leak is in
the main server loop?
Sometimes, the ssl_accept may return 0, but with an error code like
SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE, which just means that
could not complete the handshake, but not really an error (just call
ssl_accept again ... ).
But in you main server thread, you just do a continue, thus looping
again ... and creating a brand new ssl structure and socket ... you
should
loop() {
create the tcp socket
loop () {
ssl_accept from the tcp socket till non-zero
}
}
Take a look at the examples that come with openssl.
Regards,
Cesc
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
This message has been scanned for viruses by MailController -
www.MailController.altohiway.com
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: Investigating a leak
That'd do it. But if you're doing the loop in that sequence, and if you have set non-blocking on the sockets, then instead of polling for the connection you can use select or poll (depending on your platform) to wait for incoming connection activity, then enter ssl_accept in blocking mode which would probably cut down on those fake errors. Wouldn't this be a better sequence? Eww, no. Suppose the connection is aborted before you manage to call SSL_accept. The call could block indefinitely. It is almost always a mistake to use 'select' or 'poll' with blocking socket operations. DS __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Investigating a leak
http://www.openssl.org/support/faq.html Search for the words *memory leak* -Amar. Axel Andersson wrote: Hello, I'm investigating a leak in a server application, which I've managed to reduce to a test case which involves OpenSSL and threads. I've put the code up at http://www.zankasoftware.com/ssltest/server.c if anyone would be willing to take a look and see what I'm doing wrong. Every time I connect to this test with openssl s_client, it leaks some 516 KB virtual memory, which is never reclaimed. If pthread_join () is inserted after pthread_create(), no leaks occur. It does not matter if I move all the SSL calls to the reply thread, or leave the SSL_accept() call in the main thread where it is now. I'm testing this on Mac OS X with its default OpenSSL 0.9.7b. I also downloaded and built the recent 0.9.8 release and was able to reproduce the problem with that. Thanks in advance, -- Axel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] smime.p7s Description: S/MIME Cryptographic Signature
Re: Investigating a leak
On Jul 18, 2005, at 14:11, Amar Desai wrote: http://www.openssl.org/support/faq.html Search for the words *memory leak* Yes, thank you, indeed I have read that, and I am using ERR_remove_state() at the end of the thread. Looks to me like the other functions should be used before terminating the application, not for per-thread cleanup, or did I misunderstand something? -- Axel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Investigating a leak
Yes, You need to call other functions as well. -Amar Axel Andersson wrote: Yes, thank you, indeed I have read that, and I am using ERR_remove_state() at the end of the thread. Looks to me like the other functions should be used before terminating the application, not for per-thread cleanup, or did I misunderstand something? -- Axel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] smime.p7s Description: S/MIME Cryptographic Signature
RE: Investigating a leak
Hi Axel, I don't know if it helps you much, but since the FAQ is a bit cryptic it may help. I've used... ERR_remove_state() ...at the end of each thread. Plus... CRYPTO_cleanup_all_ex_data() CONF_modules_free() ERR_free_strings() EVP_cleanup() ... at the end of the application. I have no leaks that I can detect when using 0.9.7g. At the time I wrote my code I couldn't find any FAQ entry that worked along the lines of if you use function or function group , then you must call to free underlying data. My app' isn't loading external engines or doing anything fancy, so these are probably not a comprehensive list of resource releasing functions. Regards, Simon - -- Simon Edwards -Original Message- From: Amar Desai [mailto:[EMAIL PROTECTED] Sent: 18 July 2005 16:59 To: openssl-users@openssl.org Subject: Re: Investigating a leak Yes, You need to call other functions as well. -Amar Axel Andersson wrote: Yes, thank you, indeed I have read that, and I am using ERR_remove_state() at the end of the thread. Looks to me like the other functions should be used before terminating the application, not for per-thread cleanup, or did I misunderstand something? -- Axel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Investigating a leak
On Jul 18, 2005, at 18:57, Simon Edwards wrote: I don't know if it helps you much, but since the FAQ is a bit cryptic it may help. I've used... ERR_remove_state() ...at the end of each thread. Doing this. Plus... CRYPTO_cleanup_all_ex_data() CONF_modules_free() ERR_free_strings() EVP_cleanup() ... at the end of the application. Doing this now, but since my app is a server, it never really reaches the end, and the leaks build up during its lifetime, which I'd rather hoped could be more than a day or so. As it is now, the process' virtual memory size quickly reaches 3.5 GB, and then malloc() starts failing. Well, thanks for your help anyway, I appreciate it. -- Axel __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Investigating a leak
Doing this now, but since my app is a server, it never really reaches the end, and the leaks build up during its lifetime, which I'd rather hoped could be more than a day or so. As it is now, the process' virtual memory size quickly reaches 3.5 GB, and then malloc() starts failing. Something wrong is happening. Lots of folks are running apache/openssl servers for days, weeks, etc., at a time. If you're seeing memory leaks like you describe, then it is far more likely that the like is either in your code, or there's a bug in the way you're calling OpenSSL. Do you get the same growth when openssl isn't used? Make sure you xxx_free every object you xxx_new. /r$ -- Rich Salz Chief Security Architect DataPower Technology http://www.datapower.com XS40 XML Security Gateway http://www.datapower.com/products/xs40.html __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
