On Tue, Sep 22, 2009 at 11:08 AM, Yonik Seeley <yo...@lucidimagination.com> wrote:
> I'm still not sure I see the reason for complicating the IndexWriter > with warming... can't this be done just as efficiently (if not more > efficiently) in user/application space? It will be less efficient when you warm outside of IndexWriter, ie, you will necessarily delay the app's net turnaround time on being able to search newly added/deleted docs. The whole point of putting optional warming into IndexWriter was so the segment could be warmed *before* the merge commits the change to the writer's SegmentInfos. Any newly opened near-real-timer readers continue to search the old (merged away) segments, until the warming completes. This way the warming of merged segments is independent of making any newly flushed segments searchable (as long as you use CMS, or any merge scheduler that uses separate threads for merging). New segments can be flushed and then become searchable (with getReader()) even while the warming is happening. So... if your merge policy allows large merges, setting a warmer in the IndexWriter is crucial for minimizing turnaround time. But, even once you do that, merging is still IO & CPU intensive, plus IO caches are unnecessarily flushed (since we can't easily madvise/posix_fadvise from java), and we have no IO scheduler control to have merging run at very lower priority, etc., so while the merge & warming are taking place, search performance will be impacted. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
