I would suggest disabling commit from user, so you have control about how/when it will happen. ( https://stackoverflow.com/questions/15897605/disable-explicit-commit-in-solr )
And this article has info about how to configure the commit of solr: https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Em seg., 12 de abr. de 2021 às 17:12, matthew sporleder < [email protected]> escreveu: > commitWithin instead of commit would (likely) be easier since it's > just a few extra characters on the client side and might also help. > > On Mon, Apr 12, 2021 at 10:05 AM Shawn Heisey <[email protected]> wrote: > > > > On 4/12/2021 1:08 AM, Rekha Sekhar wrote: > > > We have a solrCloud setup in Kubernetes with 2 Solr instances and 3 > > > ZooKeeper instances with 1 shard. It is configured with 8G persistent > > > storage for each Solr and Zookeeper. The Memory allocated for Solr is > 16G > > > with 10G Heap size. There are a max of 2.5million records indexed. > There > > > scheduler client which will call the Solr with url - > > > /update/json?wt=json&commit=true - to do the add/update/delete > operations. > > > Occasionally there will be a huge update/delete happens with 1 million > > > records which will call the api (/update/json?wt=json&commit=true ) > with > > > 500 documents at a time, but this is called in multiple threads. > Everything > > > works fine 1 week, but suddenly we saw errors in Solr.log which makes > the > > > solr in an error state and I had to restart one of the solr node. > > > > So you're issuing a manual commit with every batch of 500 documents and > > sending those batches in parallel? > > > > That's a LOT of commit operations. Commits are just about all Solr will > > be doing with that kind of setup. Which will leave very few system > > resources for handling updates or queries. > > > > I would remove all commits from the client side and go with an automatic > > server-side setup like the following in solrconfig.xml: > > > > <autoCommit> > > <maxTime>120000</maxTime> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > <autoSoftCommit> > > <maxTime>60000</maxTime> > > </autoSoftCommit> > > > > You can look at the example solrconfig.xml files to figure out where it > > goes. In later versions of Solr I think it can be found in the > > updateHandler section. > > > > That setup will make commits happen at much more reasonable intervals, > > which might clear up the whole problem. > > > > If that setup doesn't help, a screenshot like the ones mentioned here > > can be very helpful for us to make determinations about your memory > setup: > > > > > https://cwiki.apache.org/confluence/display/solr/SolrPerformanceProblems#SolrPerformanceProblems-Askingforhelponamemory/performanceissue > > > > Thanks, > > Shawn > -- Abraços Carlos Sponchiado
