On 10/18/2017 3:09 AM, Leo Prince wrote: > Is there any known negative impacts in setting up autoSoftCommit as 1 > second other than RAM usage..?
For most users, setting autoSoftCommit to one second is a BAD idea. In many indexes, commits take longer than one second to complete. If you do heavy indexing with that setting and the commit takes longer than one second to complete, then you will end up with overlapping commits, all of which will *try* to open new a searcher. In a hypothetical situation where heavy indexing lasts for an hour and commits take enough time, Solr will be doing *constant* commits (which may overlap) for the entire hour. In that kind of scenario, Solr will prevent most of the new searchers from actually opening because of the maxWarmingSearchers configuration, so you won't see changes within the configured one second anyway. Frequent commits, especially if they overlap, will pound the CPU and Java's garbage collection, so the server is going to be extremely busy and may have difficulty handling indexing and query requests in a timely manner. Soft commits are sometimes a little bit faster than hard commits that open a new searcher, but the difference is not extreme. Later in the thread you stated "Since we are using SoftCommits, the docs written will be in RAM until a AutoCommit to reflect onto Disk". This is not *exactly* how it works. Soft commits have the *possibility* of being handled by the caching nature of the NRTCachingDirectory implementation. See the javadoc for the directory implementation: http://lucene.apache.org/core/7_0_0/core/org/apache/lucene/store/NRTCachingDirectory.html The default cache sizes stated in that documentation are 5 and 60 MB, but the actual defaults in the code are 4 and 48. If a new segment is larger than 4MB or the total amount to be cached is larger than 48MB, then it won't be saved to RAM, it'll go to disk. In a nutshell, with very heavy indexing, soft commits won't save you anything, since you're probably going to be writing some information to disk anyway. You should set autoSoftCommit to the longest possible interval you can stand for the visibility of changes. I would recommend AT LEAST 60 seconds, but if your commits finish quickly, you might be able to go lower. The autoCommit section should set openSearcher to false. Unless your autoCommit interval has been increased beyond the typical default of 15 seconds, I usually recommend making the autoSoftCommit interval longer than the autoCommit interval. You have already been given the following link. It covers most of the caveats where commits are concerned. The title of the article mentions SolrCloud, but the information applies to standalone mode too: https://lucidworks.com/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ Thanks, Shawn
