On Mon, May 01, 2006 at 04:49:27PM -0400, John Kelly wrote: > At startup, clamav-milter memory usage is approx. 11m resident and 30m > virtual. But when freshclam loads a new database, memory jumps to 20m > resident and 40m virtual. > > The changelog for 0.86 says clamav-milter scans the old database while > waiting to become idle and reload the new database. But the memory > usage looks like the old database is still in memory after the new one > is loaded. > > This is on a linux test system not processing any mail. The only > activity is loading a new database. I'm using the latest clamav code, > 0.88.2.
I am seeing the same behaviour on clamd cross-compiled for ARM/Xscale. I think this is caused by a great deal of memory fragmentation due to allocating lots of small chunks (down to 1 byte). The dynamic memory allocator is not coping with this and is not able to reuse those small chunks on database reload after freeing the old memory, so it calls brk() to increase the heap. Thats why the memory useage is moderate after loading the databse the first time, but almost doubles after reload (it stays at that heap size after reloading mutiple times though) BTW, are there any plans to reimplement the memory allocation of the database structures to allocate bigger chunks or even use a dedicated memory allocator (like BGET http://www.fourmilab.ch/bget/) ? The big advantage with this solution would be to allocate whole pages with mmap(), on which the memory for the whole database struct (and only the database) is allocated with bget, and which can be unmapped completely before reloading the database . Pages allocated with mmap and unmapped afterwards are guaranteed to be reuseable immediately, so the heap fragmentation problem could be circumvented. -- Peter Vollmer Software Engineer Innominate Security Technologies AG /device attached security/ http://www.innominate.com _______________________________________________ http://lurker.clamav.net/list/clamav-devel.html
