Thanks Shawn, Do you have any feeling for what gets traded off if we increase the maxMergeCount?
This is completely new for us because we are experimenting with indexing pages instead of whole documents. Since our average document is about 370 pages, this means that we have increased the number of documents we are asking Solr to index by a couple of orders of magnitude. (on the other hand the size of the document decreases by a couple of orders of magnitude). I'm not sure why increasing the number of documents (and reducing their size) is causing more merges. I'll have to investigate. Tom On Thu, Jul 11, 2013 at 5:29 PM, Shawn Heisey <s...@elyograg.org> wrote: > On 7/11/2013 1:47 PM, Tom Burton-West wrote: > >> We are seeing the message "too many merges...stalling" in our indexwriter >> log. Is this something to be concerned about? Does it mean we need to >> tune something in our indexing configuration? >> > > It sounds like you've run into the maximum number of simultaneous merges, > which I believe defaults to two, or maybe three. The following config > section in <indexConfig> will likely take care of the issue. This assumes > 3.6 or later, I believe that on older versions, this goes in > <indexDefaults>. > > <mergeScheduler class="org.apache.lucene.**index.** > ConcurrentMergeScheduler"> > <int name="maxThreadCount">1</int> > <int name="maxMergeCount">6</int> > </mergeScheduler> > > Looking through the source code to confirm, this definitely seems like the > case. Increasing maxMergeCount is likely going to speed up your indexing, > at least by a little bit. A value of 6 is probably high enough for mere > mortals, buy you guys don't do anything small, so I won't begin to > speculate what you'll need. > > If you are using spinning disks, you'll want maxThreadCount at 1. If > you're using SSD, then you can likely increase that value. > > Thanks, > Shawn > >
