Autowarm times will only happen when the commit has openSearcher=true or on a soft commit. But maybe your log levels aren't at INFO for the right code...
That said, your autowarm counts at 0 probably means that you're not seeing any autowarming really, so that might be a red herring. Your newSearcher event in solrconfig.xml will still be fired, but may be commented out. This is still something of a puzzle. With an index this size, your hard commits should never take more than a second or two unless you're in some very strange state. Stack traces would be in order if lengthening the commit interval doesn't work. Best, Erick On Fri, Dec 11, 2015 at 5:58 PM, Vincenzo D'Amore <v.dam...@gmail.com> wrote: > Hi All, > > an update, I have switched logging from WARN to INFO for all except for > those two: > > - org.apache.solr.core > - org.apache.solr.handler.component.SpellCheckComponent > > Well, looking at log file I'm unable to find any autowarm log line, even > after few updates and commits. > > Looking at solrconfig.xml I see most autowarmCount parameters are set to 0 > > <filterCache class="solr.FastLRUCache" size="1500" initialSize="1000" > autowarmCount="0" /> > <queryResultCache class="solr.LRUCache" size="8096" initialSize="1024" > autowarmCount="0" /> > <documentCache class="solr.FastLRUCache" size="10240" initialSize="4096" > autowarmCount="0" /> > <cache name="perSegFilter" class="solr.search.LRUCache" size > ="10" initialSize="0" autowarmCount="10" regenerator="solr.NoOpRegenerator" > /> > > Not sure what this means... > > On Sat, Dec 12, 2015 at 1:13 AM, Vincenzo D'Amore <v.dam...@gmail.com> > wrote: > >> Thanks Erick, Mark, >> >> I'll raise maxTime asap. >> Just to be sure understand, given that I have openSearcher=false, I >> suppose it shouldn't trigger autowarming at least until a commit is >> executed, shouldn't it? >> >> Anyway, I don't understand, given that maxTime is very aggressive, why >> hard commit takes so long. >> >> Thanks again for your answers. >> Vincenzo >> >> >> On Fri, Dec 11, 2015 at 7:22 PM, Erick Erickson <erickerick...@gmail.com> >> wrote: >> >>> First of all, your autocommit settings are _very_ aggressive. Committing >>> every second is far to frequent IMO. >>> >>> As an aside, I generally prefer to omit the maxDocs as it's not all >>> that predictable, >>> but that's a personal preference and really doesn't bear on your problem.. >>> >>> My _guess_ is that you are doing a lot of autowarming. The number of docs >>> doesn't really matter if your autowarming is taking forever, your Solr >>> logs >>> should report the autowarm times at INFO level, have you checked those? >>> >>> The commit settings shouldn't be a problem in terms of your server dying, >>> the indexing process flushes docs to the tlog independent of committing so >>> upon restart they should be recovered. Here's a blog on the subject: >>> >>> >>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ >>> >>> Best, >>> Erick >>> >>> On Fri, Dec 11, 2015 at 8:24 AM, Vincenzo D'Amore <v.dam...@gmail.com> >>> wrote: >>> > Hi all, >>> > >>> > I have a SolrCloud cluster with a collection (2.5M docs) with 3 shards >>> and >>> > 15 replicas. >>> > There is a solrj application that feeds the collection, updating few >>> > documents every hour, I don't understand why, at end of process, the >>> hard >>> > commit takes about 8/10 minutes. >>> > >>> > Even if there are only few hundreds of documents. >>> > >>> > This is the autocommit configuration: >>> > >>> > <autoCommit> >>> > <maxDocs>10000</maxDocs> >>> > <maxTime>1000</maxTime> >>> > <openSearcher>false</openSearcher> >>> > </autoCommit> >>> > >>> > In your experience why hard commit takes so long even for so few >>> documents? >>> > >>> > Now I'm changing the code to softcommit, calling commit (waitFlush = >>> > false, waitSearcher >>> > = false, softCommit = true); >>> > >>> > solrServer.commit(false, false, true);. >>> > >>> > I have configured NRTCachingDirectoryFactory, but I'm a little bit >>> worried >>> > if a server goes down (something like: kill -9, SolrCloud crashes, out >>> of >>> > memory, etc.), and if, using this strategy >>> softcommit+NRTCachingDirectory, >>> > SolrCloud instance could not recover a replica. >>> > >>> > Should I worry about this new configuration? I was thinking to take a >>> > snapshot of everything every day, in order to recover immediately the >>> > index. Could this be considered a best practice? >>> > >>> > Thanks in advance for your time, >>> > Vincenzo >>> > >>> > -- >>> > Vincenzo D'Amore >>> > email: v.dam...@gmail.com >>> > skype: free.dev >>> > mobile: +39 349 8513251 >>> >> >> >> >> -- >> Vincenzo D'Amore >> email: v.dam...@gmail.com >> skype: free.dev >> mobile: +39 349 8513251 >> > > > > -- > Vincenzo D'Amore > email: v.dam...@gmail.com > skype: free.dev > mobile: +39 349 8513251
