I am looking all the time to LUCENE-831, which is a new version of FieldCache that is compatible with IndexReader.reopen() and invalidates only reloaded segments. In each release of Lucene I am very unhappy, because it is still not in. The same problem like yours is if you have a one million documents index that is updated by adding a few documents each half hour. If you use sorting by a field, whenever the index is reopened and you really only a very small segment is added, nevertheless the complete FieldCache is rebuild, very bad :(.
So I think the ultimative fix would be to hopefully apply LUCENE-831 soon and also use LUCENE-1461 as RangeFilter cache. ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: [EMAIL PROTECTED] ________________________________________ From: robert engels [mailto:[EMAIL PROTECTED] Sent: Thursday, December 04, 2008 9:39 PM To: java-dev@lucene.apache.org Subject: Re: [jira] Commented: (LUCENE-855) MemoryCachedRangeFilter to boost performance of Range queries I can't seem to post to Jira, so I am attaching here... I attached QueryFilter.java. In reading this patch, and other similar ones, the problem seems to be that if the index is modified, the cache is invalidated, causing a complete reload of the cache. Do I have this correct? The attached patch works really well in a highly interactive environment, as the cache is only invalidated at the segment level. The MyMultiReader is a subclass that allows access to the underlying SegmentReaders. The patch cannot be applied, but I think the implementation works far better in many cases - it is also far less memory intensive. Scanning the bitset could also be optimized very easily using internal skip values. Maybe this is completely off-base, but the solution has worked very well for us. Maybe this is a completely different issue and separate incident should be opened ? is there any interest in this? --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
