He has waitSearcher as false it looks, so all the time should be in the commit. So that amount of time does sound odd.
I would certainly change those commit settings though. I would not use maxDocs, that is an ugly way to control this. And one second is much too aggressive as Erick says. If you want to attempt that kind of visibility, you should use the softAutoCommit. The regular autoCommit should be at least 15 or 20 seconds. - Mark On Fri, Dec 11, 2015 at 1:22 PM Erick Erickson <erickerick...@gmail.com> wrote: > First of all, your autocommit settings are _very_ aggressive. Committing > every second is far to frequent IMO. > > As an aside, I generally prefer to omit the maxDocs as it's not all > that predictable, > but that's a personal preference and really doesn't bear on your problem.. > > My _guess_ is that you are doing a lot of autowarming. The number of docs > doesn't really matter if your autowarming is taking forever, your Solr logs > should report the autowarm times at INFO level, have you checked those? > > The commit settings shouldn't be a problem in terms of your server dying, > the indexing process flushes docs to the tlog independent of committing so > upon restart they should be recovered. Here's a blog on the subject: > > > https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/ > > Best, > Erick > > On Fri, Dec 11, 2015 at 8:24 AM, Vincenzo D'Amore <v.dam...@gmail.com> > wrote: > > Hi all, > > > > I have a SolrCloud cluster with a collection (2.5M docs) with 3 shards > and > > 15 replicas. > > There is a solrj application that feeds the collection, updating few > > documents every hour, I don't understand why, at end of process, the hard > > commit takes about 8/10 minutes. > > > > Even if there are only few hundreds of documents. > > > > This is the autocommit configuration: > > > > <autoCommit> > > <maxDocs>10000</maxDocs> > > <maxTime>1000</maxTime> > > <openSearcher>false</openSearcher> > > </autoCommit> > > > > In your experience why hard commit takes so long even for so few > documents? > > > > Now I'm changing the code to softcommit, calling commit (waitFlush = > > false, waitSearcher > > = false, softCommit = true); > > > > solrServer.commit(false, false, true);. > > > > I have configured NRTCachingDirectoryFactory, but I'm a little bit > worried > > if a server goes down (something like: kill -9, SolrCloud crashes, out of > > memory, etc.), and if, using this strategy > softcommit+NRTCachingDirectory, > > SolrCloud instance could not recover a replica. > > > > Should I worry about this new configuration? I was thinking to take a > > snapshot of everything every day, in order to recover immediately the > > index. Could this be considered a best practice? > > > > Thanks in advance for your time, > > Vincenzo > > > > -- > > Vincenzo D'Amore > > email: v.dam...@gmail.com > > skype: free.dev > > mobile: +39 349 8513251 > -- - Mark about.me/markrmiller
