Thanks Erick, Mark, I'll raise maxTime asap. Just to be sure understand, given that I have openSearcher=false, I suppose it shouldn't trigger autowarming at least until a commit is executed, shouldn't it?
Anyway, I don't understand, given that maxTime is very aggressive, why hard commit takes so long. Thanks again for your answers. Vincenzo On Fri, Dec 11, 2015 at 7:22 PM, Erick Erickson <erickerick...@gmail.com> wrote: > First of all, your autocommit settings are _very_ aggressive. Committing > every second is far to frequent IMO. > > As an aside, I generally prefer to omit the maxDocs as it's not all > that predictable, > but that's a personal preference and really doesn't bear on your problem.. > > My _guess_ is that you are doing a lot of autowarming. The number of docs > doesn't really matter if your autowarming is taking forever, your Solr logs > should report the autowarm times at INFO level, have you checked those? > > The commit settings shouldn't be a problem in terms of your server dying, > the indexing process flushes docs to the tlog independent of committing so > upon restart they should be recovered. Here's a blog on the subject: > > > https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > Best, > Erick > > On Fri, Dec 11, 2015 at 8:24 AM, Vincenzo D'Amore <v.dam...@gmail.com> > wrote: > > Hi all, > > > > I have a SolrCloud cluster with a collection (2.5M docs) with 3 shards > and > > 15 replicas. > > There is a solrj application that feeds the collection, updating few > > documents every hour, I don't understand why, at end of process, the hard > > commit takes about 8/10 minutes. > > > > Even if there are only few hundreds of documents. > > > > This is the autocommit configuration: > > > > <autoCommit> > > <maxDocs>10000</maxDocs> > > <maxTime>1000</maxTime> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > In your experience why hard commit takes so long even for so few > documents? > > > > Now I'm changing the code to softcommit, calling commit (waitFlush = > > false, waitSearcher > > = false, softCommit = true); > > > > solrServer.commit(false, false, true);. > > > > I have configured NRTCachingDirectoryFactory, but I'm a little bit > worried > > if a server goes down (something like: kill -9, SolrCloud crashes, out of > > memory, etc.), and if, using this strategy > softcommit+NRTCachingDirectory, > > SolrCloud instance could not recover a replica. > > > > Should I worry about this new configuration? I was thinking to take a > > snapshot of everything every day, in order to recover immediately the > > index. Could this be considered a best practice? > > > > Thanks in advance for your time, > > Vincenzo > > > > -- > > Vincenzo D'Amore > > email: v.dam...@gmail.com > > skype: free.dev > > mobile: +39 349 8513251 > -- Vincenzo D'Amore email: v.dam...@gmail.com skype: free.dev mobile: +39 349 8513251
