A merge shouldn't make the machine completely non-responsive, just, slower to run searches / index documents.
What kind of machine / IO system is this? You can set maxMergeDocs to limit how large the merge is allowed to be. But, be careful, since if you set this too small you'll wind up with way too many segments over time, which'll slow down search and risk file handle exhaustion. Likewise, increase mergeFactor. You could also try decreasing the merge thread priority in ConcurrentMergeScheduler... though that risk starvation of the merge thread if your CPUs are really saturated doing indexing/searching. Another thing to try is the BalancedSegmentMergePolicy (in contrib/misc); it also tries to avoid big merges. Also, how are you opening new readers? Can you share more how you are using Lucene? Mike On Tue, Dec 22, 2009 at 5:57 PM, Siraj Haider <si...@jobdiva.com> wrote: > Hi Mike, > You are right, sometimes there is an implicit merge running when the machine > goes non-responsive. How can we avoid running those merges during the day > and how can we minimize the effect it will have on searches? > > -siraj > > Michael McCandless wrote: >> >> Is it possible a large merge is running? >> >> You can turn on IndexWriter.setInfoStream to see more details about >> what IW is doing, including merging. >> >> Mike >> >> On Tue, Dec 22, 2009 at 5:19 PM, Siraj Haider <si...@jobdiva.com> wrote: >> >>> >>> Hello guys, >>> We have a dilemma on a few of our lucene machines. We have a tomcat >>> running >>> our servlets for searching and indexing on each of these machines. Its a >>> live index where documents are being added to index while online searches >>> are also being served at the same time. Indexing happens every 5 minutes >>> and if there are new documents added, the index gets reopend. For most >>> of >>> the times the performance is very good, but under heavy load of searches, >>> the machine goes non-responsive. We can still telnet to machine and see >>> that cpu-wise its not bad, but I/O seems to be a problem. Is there >>> anything >>> we might be doing to cause it or anything that we can do to avoid it. I >>> know I did not provide a lot of information about how we are indexing and >>> searching but I will answer any question anyone might have. >>> >>> thanks in advance >>> -siraj >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
