On 8/17/2015 7:04 AM, Maulin Rathod wrote:
> We have observed that Intermittently querying become slower when 
> documentCache become empty. The documentCache is getting flushed whenever new 
> document added to the collection.
> 
> Is there any way by which we can ensure that newly added documents are 
> visible without losing data in documentCache? We are trying to use soft 
> commit but it also flushes all documents in documentCache.

<snip>

> <autoSoftCommit>
>       <maxTime>50</maxTime>
> </autoSoftCommit>

You are doing a soft commit within 50 milliseconds of adding a new
document.  Solr can have severe performance problems when autoSoftCommit
is set to 1000 -- one second.  50 milliseconds is one twentieth of a
very low value that is known to cause problems.  It can make the problem
much more than 20 times worse.

Please read this article:

http://lucidworks.com/blog/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/

Note one particular section, which says the following:  Don’t listen to
your product manager who says "we need no more than 1 second latency".

You need to set your commit interval as long as you possibly can.  I
personally wouldn't go longer than 60 seconds, 30 seconds if the commits
complete particularly fast.  It should be several minutes if that will
meet your needs.  When your commit interval is very low, Solr's caches
can become useless, as you've noticed.

TL;DR info:  Your autoCommit settings have openSearcher set to false, so
they do not matter for the problem you have described. I would probably
increase that to 5 minutes rather than 15 seconds, but that is not very
important here, and 15 seconds for hard commits that don't open a new
searcher is known to have a low impact on performance.  "Low" impact
isn't the same as NO impact, so I keep this interval long as well.

Thanks,
Shawn

Reply via email to