On Sat, Jan 23, 2010 at 08:25:04PM -0600, Peter Karman wrote: > I have a collection of about 90k+ docs totaling about 2.2G. > > When I index with KS 0.30_072 I see memory use climb to about 80M and hold > relatively steady. > > With svn trunk, I see memory use climb steadily throughout the 15 minutes > or so it takes to index, reaching 600M+.
We gots us a memory leak, then. Not surprising. It's hard to avoid making refcounting errors while developing. That's why we have test_valgrind... only the suppressions file trunk/devel/conf/kinoperl.supp has been wrong since the switch to LockFreeRegistry, so test_valgrind is currently broken. > Initially I thought this was due to the recent changes in trunk to > SortExternal and so didn't think much of it since I knew that feature was > currently receiving some attention to reduce memory use. But I didn't want > possible other leaks to go unreported. > > So I guess my question is: is that kind of memory use to be expected? If you have sortable fields, then high memory usage is to be expected until we fix SortWriter. However, SortWriter hasn't changed fundamentally in between 0.30_72 and now. So it would seem that this is a leak, almost certainly due to a refcounting error. Marvin Humphrey
