First off I'm not all that familiar with the openSSL code base so my comment may be inappropriate.
A couple years ago I made a post that I felt the memory managment is not done optimally. I think what you have discovered illustrates this. IMHO we should have a higher level memory managment layer between the library level functions and the actual system malloc() etc. routines. What I am thinking is that we need to define a level where teh underlying system malloc()s are done one or more pages at a time and that the pages are logically associated with a connection. The idea then is that any memory needed for a specific connection gets allocated in the page pool associated with that connection. If we do this, then when the connection is closed, the pool can be re-initialized and this will erase any leaks. The code to do something like this is conceptually quite easy to write... I've written some stubs for it - but they are still quite primative. I note that you show 53,000+ allocations. Malloc is intended to conserve memory and thus does lots of nice things like searching for an appropriate sized chunk in the free space list and so forth. While this is appropriate for many situations it means there is a considerable overhead associated with doing a large number of malloc()'s. Similarly there is a considerable overhead with the free()s. Most of this can be avoided by setting up a pool of pages of memory associated with a connection. There is an additional benefit. If the connections are initiated in a random fashion and the processing of each connection is interleaved, then the system malloc() approach scatters the underlying chunks of memory willy nilly all over the place in the memory of the processor. This means that the virtual memory managment system is emasculated. On the other hand, if we set up a pool of pages associated with a connection then if that connection stalls the virtual memory manager has a chance to flip specifically those pages out to disk. The idea here is that all memory for a given connection has to live in a specific group of pages and that memory for some other connection is excluded from these pages. If you feel these ideas have merit - then I'll be happy to do some work on this. I do have this feeling that it will be relatively easy to start layering this approach into the existing code base in an upward compatible fashion. On Tue, Mar 23, 2004 at 09:05:12AM -0600, Avery, Ken wrote: > I have narrowed it down to the function BN_BLINDING_new in the file > crypto\bn\bn_blind.c, the memory allocated for the BN_BLINDING structure > never gets freed. I am assuming that the BIGNUM structures allocated > with BN_new inside of BN_BLINDING never gets freed also. > > > > Here are my test results after running 24 hours monitoring the > OPENSSL_malloc and OPENSSL_free calls: > > 1. BN_BLINDING - allocations 53,615, frees 0, outstanding 53,615 > 2. BN_new - allocations 8,347,200, frees 8,127,872 outstanding > 219,328 > 3. I also track the heap and it grows proportional to the lack of > BN frees > > > > Is there anyone out there willing to help out that understands the big > number code? Can anyone at least tell me if Apache/mod_ssl/OpenSSL needs > to initialize some kind of callback? Like CRYPTO_set_locking_callback <- > Just an example, this is in the code, are there any other callbacks > anyone can think of to get the OpenSSL code to release the BN memory? > > > > > ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
