On 02/17/2014 05:38 PM, Shawn Heisey wrote:
On 2/17/2014 6:06 PM, Colin Bartolome wrote:
We're using Solr version 4.2.1, in case new functionality has helped
with this issue.
We have our Solr servers doing automatic soft commits with maxTime=1000.
We also have a scheduled job that triggers a hard commit every fifteen
minutes. When one of these hard commits happens while a soft commit is
already in progress, we get that ubiquitous warning:
PERFORMANCE WARNING: Overlapping onDeckSearchers=2
Recently, we had an occasion to have a second scheduled job also issue a
hard commit every now and then. Since our maxWarmingSearchers value was
set to the default, 2, we occasionally had a hard commit trigger when
two other searchers were already warming up, which led to this:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers
available to handle this request
as the servers started responded with a 503 HTTP response.
It seems like automatic soft commits wait until the hard commits are out
of the way before they proceed. Is there a way to do the same for hard
commits? Since we're passing waitSearcher=true in the update request
that triggers the hard commits, I would expect the request to block
until the server had enough headroom to service the commit. I did not
expect that we'd start getting 503 responses.
Remember this mantra: Hard commits are about durability, soft commits
are about visibility. You might already know this, but it is the key to
figuring out how to handle commits, whether they are user-triggered or
done automatically by the server.
With Solr 4.x, it's best to *always* configure autoCommit with
openSearcher=false. This does a hard commit but does not open a new
searcher. The result: Data is flushed to disk and the current
transaction log is closed. New documents will not be searchable after
this kind of commit. For maxTime and maxDocs, pick values that won't
result in huge transaction logs, which increase Solr startup time.
http://wiki.apache.org/solr/SolrPerformanceProblems#Slow_startup
For document visibility, you can rely on autoSoftCommit, and you
indicated that you already have it configured. Decide how long you can
wait for new content that has just been indexed. Do you *really* need
new data to be searchable within one second? If so, you're good. If
not, increase the maxTime value here. Be sure to make the value at
least a little bit longer than the amount of time it takes for a soft
commit to finish, including cache warmup time.
Thanks,
Shawn
Increasing the maxTime value doesn't actually solve the problem, though;
it just makes it a little less likely. Really, the soft commits aren't
the problem here, as far as we can tell. It's that a request that
triggers a hard commit simply fails when the server is already at
maxWarmingSearchers. I would expect the request to queue up and wait
until the server could handle it.
