Re: Custom sorting - memory leaks

2006-08-23 Thread Chris Hostetter
: Is it possible to add some kind of SortComparatorSource class / not : interface in new lucene releases with default behaviour to cache : ScoreDocComparators? Anything's possible with a well written patch :) : > On a related note: have you considered using FunctionQueries (in teh Solr : > code

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Chris, I see what you're saying now ... yes, for cases like this it probably would be useful to a way to prevent the Comparator from being cached ... That's what I'm talking about. I agree this is very uncommon case. perhaps by adding a SortComparatorSource.isCachable() method ... but the cha

Re: Custom sorting - memory leaks

2006-08-21 Thread Chris Hostetter
: > DistanceComparatorSource example in LIA). Assume that the 'specified : > coordinate' is different for each query. This means a new custom comparator : > must be created for each query, which is ok. However, Lucene caches the : Lets suppose another custom sortings - random, using dynamically :

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
I forgot to mention that the query threads are created only once. Peter On 8/21/06, Peter Keegan <[EMAIL PROTECTED]> wrote: Aleksey, My app has precise and predictable control over when the index reader/searcher gets refreshed. When this occurs, the sorting values for all the docs in the new i

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
Aleksey, My app has precise and predictable control over when the index reader/searcher gets refreshed. When this occurs, the sorting values for all the docs in the new index are generated by the app and cached. Each query thread has it's own ScoreDocComparator which contains a reference to the s

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Peter, As i understand correctly, implementing the equals and hashCode methods let us avoid caching the same sorting values, i.e. if we have a few points to calc distance from then we can implement equals and hashCode methods based on point value and we'll have one cached comparator per different

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
I avoided the cache by implementing the equals and hashCode methods appropriately in my custom ScoreDocComparator as described here: http://www.lucenebook.com/blog/errata/2006/03/01/memory_leak.html Peter On 8/21/06, Aleksey Serba <[EMAIL PROTECTED]> wrote: Hi Chris, On 5/17/06, Peter Keegan

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Hi Chris, On 5/17/06, Peter Keegan wrote: Suppose I have a custom sorting 'DocScoreComparator' for computing distances on each search hit from a specified coordinate (similar to the DistanceComparatorSource example in LIA). Assume that the 'specified coordinate' is different for each query. Thi

Re: Custom sorting - memory leaks

2006-08-18 Thread Chris Hostetter
: You can reproduce OutOfMemory easily. I've attach test files - this is : altered DistanceSortingTest example from LIA book. Also you can : profile it and see caching of distances arrays. An OutOfMemory error is differnet from a memory leak. Sorting with a custom Comparator does in fact use a l

Custom sorting - memory leaks

2006-08-18 Thread Aleksey Serba
Hi! Could you please read the following discussion in java-user mail list - http://www.gossamer-threads.com/lists/lucene/java-user/35352 You can reproduce OutOfMemory easily. I've attach test files - this is altered DistanceSortingTest example from LIA book. Also you can profile it and see cachi