Yep, that assumes you can afford it to be 5 minutes between the time you send a doc to Solr and your users can search it.
Part of it depends on what your indexing rate is. If you’re only sending docs occasionally, you may want to make that longer. Frankly, though, the interval there isn’t too important practically, it’s opening a new searcher that impacts your setup most obviously. Best, Erick > On Jul 21, 2020, at 3:52 PM, Tyrone Tse <tyrone...@hotmail.com> wrote: > > Eric > > Thanks for your quick response. > So in the solrconfig.xml keep the out of the box setting of 15 seconds > > <autoCommit> > <maxTime>15000</maxTime> > <openSearcher>false</openSearcher> > </autoCommit> > > and also have the <autoSoftCommit> setting set to something like 5 minutes > <autoSoftCommit> > <maxTime>300000</maxTime> > </autoSoftCommit> > > Then in the existing SolrJ code just simply delete the line > > up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true); > > Is this what you recommended I try. > > Thanks > > Tyrone > > > On Tue, Jul 21, 2020 at 1:16 PM Erick Erickson <erickerick...@gmail.com> > wrote: > >> What you’re seeing is the input from the client. You’ve passed true, true, >> which are waitFlush and waitSearcher >> which sets softCommit _for that call_ to false. It has nothing to do with >> the settings in your config file. >> >> bq. I am not passing the parameter to do a softCommit in the SolrJ >> command. >> >> I don’t think so. That’s a hard commit. This is a little tricky since the >> waitFlush and waitSearcher params don’t >> tell you that they are about hard commits. There used to only be hard >> commits, so... >> >> But…. these settings are highly suspicious. Here’s the long form: >> >> >> https://lucidworks.com/post/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >> >> It is risky to have your autocommit settings commented out. You risk >> transaction logs growing >> forever to no purpose whatsoever. >> >> Your call does a hard commit _and_ opens a new searcher for, apparently, >> every document. >> >> But your autoSoftCommit settings also open a new searcher without doing >> anything about flushing >> data to disk every second. Usually, this is far too often unless you have >> extremely stringent latency >> requirements, and in this case unless you’re only indexing once in a great >> while your caches >> are pretty useless. >> >> I strongly urge you to uncomment autocommit settings. Make the autoCommit >> interval something >> reasonable (15-60 seconds for instance) with openSearcher=false. >> >> Then lengthen your autoSoftCommit settings to as long as you can stand. >> The longer the interval, >> the less work you’ll do opening new searchers, which is a rather expensive >> operation. I like >> 5-10 minutes if possible, but your app may require shorter intervals. >> >> Then don’t send any commit settings in your SolrJ program at all. >> >> Best, >> Erick >> >>> On Jul 21, 2020, at 1:32 PM, Tyrone Tse <tyrone...@hotmail.com> wrote: >>> >>> I am using Solr 8.5 cloud, and in my collection I have edited the >>> solrconfig.xml file to use >>> <autoSoftCommit> >>> <maxTime>1000</maxTime> >>> </autoSoftCommit> >>> >>> and commented out the default <autoCommit> configuration >>> >>> <!-- >>> <autoCommit> >>> <maxTime>15000</maxTime> >>> <openSearcher>false</openSearcher> >>> </autoCommit> >>> --> >>> >>> We are using SolrJ to post files to the Solr here is the snippet of Java >>> code that does it >>> >>> try(HttpSolrClient solrClient = solr.build()){ >>> ContentStreamUpdateRequest up = new >>> ContentStreamUpdateRequest("/update/extract"); >>> up.addFile(f, mimeType); >>> String tempId = f.getName() + (new Date()).toString(); >>> up.setParam("literal.id", tempId); >>> up.setParam("literal.username", user); >>> up.setParam("literal.fileName", f.getName()); >>> up.setParam("literal.filePath", path); >>> up.setParam("uprefix", "attr_"); >>> up.setParam("fmap.content", "attr_content"); >>> up.setAction(AbstractUpdateRequest.ACTION.COMMIT, true, true); >>> logger.info("PreRequest"); >>> solrClient.request(up); >>> logger.info("PostRequest"); >>> resultId = tempId; >>> } catch (IOException | SolrServerException | >>> HttpSolrClient.RemoteSolrException e) { >>> logger.error("Error connecting.committing to Solr", e); >>> } >>> >>> So I am not passing the parameter to do a softCommit in the SolrJ >> command. >>> >>> When I posted a file to my Solr core, when I look at the solr.log file I >>> see the following information >>> >>> 2020-07-21 16:38:54.719 INFO (qtp1546693040-302) [c:files s:shard1 >>> r:core_node5 x:files_shard1_replica_n2] >> o.a.s.u.p.LogUpdateProcessorFactory >>> [files_shard1_replica_n2] webapp=/solr path=/update >>> >> params={update.distrib=TOLEADER&update.chain=files-update-processor&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from= >>> http://192.168.1.191:8983/solr/files_shard2_replica_n6/ >>> >>> Does having <autoSoftCommit> set in the solrconfig.xml override REST >> Post >>> calls that have the parameter softCommit=false and force a softCommit >> when >>> the data is posted to Solr. >>> >>> Thanks in advance. >> >>
