Just to re-iterate Charles' response with an example, we have a system which needs to be as Near RT as we can make it. So we have application level commitWith set to 250ms. Yes, we have to turn off a lot of caching, auto-warming, etc, but it was necessary to make the index as real time as we needed it to be. Now we have the benefit of being able to throw a lot of hardware, RAM and SSDs at this in order to get any kind of sane search latency.
We have the luxury of being able to afford that, but it comes with other problems because we have an index that is changing so fast (replicating to other nodes in the cloud becomes tricky, peer sync fails most of the time, etc.) What is your use case that requires this level of real-time access? On 15 July 2015 at 13:59, Reitzel, Charles <charles.reit...@tiaa-cref.org> wrote: > And, to answer your other question, yes, you can turn off auto-warming. > If your instance is dedicated to this client task, it may serve no purpose > or be actually counter-productive. > > In the past, I worked on a Solr-based application that committed > frequently under application control (vs. auto commit) and we turned off > all auto-warming and most of the caching. > > There is scant documentation in the new Solr reference (cwiki.apache.org), > but the old docs cover this well and appear current enough: > https://wiki.apache.org/solr/SolrCaching > > Just a thought: would <useColdSearcher>true</useColdSearcher> be helpful > here? > > Also, since you have just inserted the documents, it sounds like you > probably could search by ID ... > > -----Original Message----- > From: Shawn Heisey [mailto:apa...@elyograg.org] > Sent: Tuesday, July 14, 2015 6:04 PM > To: solr-user@lucene.apache.org > Subject: Re: SOLR nrt read writes > > On 7/14/2015 12:19 PM, Bhawna Asnani wrote: > > I have a use case where we have to write data into solr and > > immediately read it back. > > The read is not get by Id but a search call. > > > > I am doing a softCommit after every such write which needs to be > > visible immediately. > > However sometimes the changes are not visible immediately. > > > > We have a solr cloud but I have also tried sending reads, writes and > > commits to cloud leader only and still there is some latency. > > > > Has anybody tried to use solr this way? > > Don't ignore what Erick has said just because you're getting this reply > from someone else. That advice is correct. My intent here is to provide > more detail. > > Since you are not doing a retrieval by id (uniqueKey field), you can't use > the Realtime Get handler. That handler would get the latest version of a > doc, whether it has been committed or not. The transaction logs > (configured with updateLog in solrconfig.xml) are used to retrieve > uncommitted information. Can you change your retrieval so it's by id > rather than a search query? If you can, that might solve this for you. > > Normally, if you do a commit operation with openSearcher=true and > waitSearcher=true, control of the program will not be returned until that > commit is completely done ... but as Erick said, if you are doing a LOT of > commits very quickly, you're probably going to exceed maxWarmingSearchers, > and in that scenario, you cannot rely on using the commit operation as a > blocker for your retrieval attempt. > > In order to have any hope of getting what you want with your current > methods, your commit frequency must be low enough that each commit has time > to finish before the next one begins. I personally would not do commits > more often than once a minute. Commits on my larger index shards are known > to take up to ten seconds when the index is quiet, and even more if the > index is busy. There are ways to make commits happen faster, but it often > involves disabling features that you might want to leave enabled. > > Thanks, > Shawn > > > ************************************************************************* > This e-mail may contain confidential or privileged information. > If you are not the intended recipient, please notify the sender > immediately and then delete it. > > TIAA-CREF > ************************************************************************* >
