Related: maxWarmingSearchers behavior was fixed (block for another commit to succeed first rather than fail) in Solr 6.4 and later. https://issues.apache.org/jira/browse/SOLR-9712
Also, if any of your "realtime" search requests only involve retrieving certain documents by ID, then you can use "realtime get" without opening a new searcher. -Yonik On Tue, Oct 17, 2017 at 9:45 AM, Leo Prince <leoprince.discussi...@gmail.com> wrote: > Hi, > > Thank you Emir, Erick and Shawn for your inputs. > > I am currently using SolrCloud and planning to try out commitWithin > parameter to reduce hard commits as per your advise. Though, just wanted to > double check whether commitWithin have any > negative impacts in SolrCloud environment like lag to search from other > nodes in SolrCloud. > > Thanks, > Leo Prince. > > On Tue, Oct 17, 2017 at 4:01 AM, Shawn Heisey <apa...@elyograg.org> wrote: > >> I'm supplementing the other replies you've already gotten. See inline: >> >> >> On 10/13/2017 2:30 AM, Leo Prince wrote: >> > I am getting the following errors/warnings from Solr > > 1, ERROR: > >> org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > >> > Error opening new searcher. exceeded limit of maxWarmingSearchers=2, >> > try again later. 2, PERFORMANCE WARNING: Overlapping > onDeckSearchers=2 >> 3, WARN: DistributedUpdateProcessor error sending >> See this FAQ entry: >> >> https://wiki.apache.org/solr/FAQ?highlight=%28ondecksearchers%29#What_ >> does_.22exceeded_limit_of_maxWarmingSearchers.3DX.22_mean.3F >> >> > So my concern is, is there any chance of performance issues when > >> number of commits are high at a particular point of time. In our > >> application, we are approximating like 100-500 commits can happen > >> simultaneously from application and autocommit too for those > >> individual requests which are not committing individually after the > >> write. > > Autocommit is configured as follows, > > <autoCommit> >> <maxTime>15000</maxTime> > <openSearcher>false</openSearcher> >> </autoCommit> >> The commits generated by this configuration are not opening new >> searchers, so they are not connected in any way to the error messages >> you're getting, which are about new searchers. Note that this >> particular kind of configuration is strongly recommended for ALL Solr >> installs using Solr 4.0 and later, so that transaction logs do not grow >> out of control. I would personally use a value of at least 60000 for >> autoCommit, but there is nothing wrong with a 15 second interval. >> >> The initial reply you got on this thread mentioned that commits from the >> application are discouraged. I don't agree with this statement, but I >> will say that the way that people *use* commits from the application is >> frequently very wrong, and because of that, switching to fully automatic >> soft commits is often the best solution, because they are somewhat >> easier to control. >> >> We have no way of knowing how long it will take to open a new searcher >> on your index. It depends on a lot of factors. Whatever that time is, >> commits should not be happening on a more frequent basis than that >> interval. They should happen *less* frequently than that interval if at >> all possible. Depending on exactly how Solr is configured, it might be >> possible to reduce the amount of time that a commit with a new searcher >> takes to complete. >> >> Definitely avoid sending a commit after every document. It is generally >> also a bad idea to send a commit with every update request. If you want >> to do commits manually, then you should index a bunch of data and then >> send one commit to make all those changes visible, and not do another >> commit until you do another batch of indexing. >> >> Thanks, >> Shawn >> >>
