Hi, We have been using Solr for last three years without any problem.
We have recently migrated our servers to a new datacenter, and our servers have been upgraded as well. We also used this opportunity to upgrade the software we use in our infrastructure. As a result of this, Solr is upgraded to version 3.6. However, we have been experiencing issues while merging. We have about 130.000 documents, and average page count is something about 80 pages. We are running a delta indexing every hour, and about 20.000-30.000 documents get re-indexed. The reason is that we have about 80k unique visitors, and we need to re-index deleted, added and viewed documents (for view count). We are considering to create a new indexing format to separate viewed documents, and run it daily to reduce number of documents for the hourly operation. However it is not going to solve the problem, it will just make things better. (We have workarounds right now to prevent the issue, but no real solution) We have no problems while indexing. However, when the merging starts, we simply lose the connection to the server, and solr becomes unresponsive for 7-12 minutes. Average load in the server also increases. So far; - We tried changing compound file format - We tried TieredMergePolicy, however we haven't changed maxMergeAtOnce and segmentsPerTier. If they are going to help us, what should be the ideal value? - We switched back to LogByteSizeMergePolicy since it was the default value in our old configuration but it didn't help either. What do you suggest? How can we prevent the unresponsiveness during merging. The reason I am saying merging is the problem is that we experience the issue right after indexing, and it takes only 7-12 minutes. Could be something internal I do not know though. However, the new server is quite powerful, way powerful than old server. So, it was a bit of surprise since we haven't had any issues in the old setup. Thanks,