On Tue, Oct 23, 2012 at 3:07 PM, Erick Erickson <erickerick...@gmail.com> wrote: > Maybe you've been looking at it but one thing that I didn't see on a fast > scan was that maybe the commit bit is the problem. When you commit, > eventually the segments will be merged and a new searcher will be opened > (this is true even if you're NOT optimizing). So you're effectively committing > every 1-2 seconds, creating many segments which get merged, but more > importantly opening new searchers (which you are getting since you pasted > the message: Overlapping onDeckSearchers=2). > > You could pinpoint this by NOT committing explicitly, just set your autocommit > parameters (or specify commitWithin in your indexing program, which is > preferred). Try setting it at a minute or so and see if your problem goes away > perhaps? > > The NRT stuff happens on soft commits, so you have that option to have the > documents immediately available for search. >
Thanks, Erick. I'll play around with different configurations. So far just removing the periodic optimize command worked wonders. I'll see how much it helps or hurts to run that daily or more or less frequent. -- Dotan Cohen http://gibberish.co.il http://what-is-what.com
