Hi All,
an update, I have switched logging from WARN to INFO for all except for
those two:
- org.apache.solr.core
- org.apache.solr.handler.component.SpellCheckComponent
Well, looking at log file I'm unable to find any autowarm log line, even
after few updates and commits.
Looking at solrconfig.xml I see most autowarmCount parameters are set to 0
<filterCache class="solr.FastLRUCache" size="1500" initialSize="1000"
autowarmCount="0" />
<queryResultCache class="solr.LRUCache" size="8096" initialSize="1024"
autowarmCount="0" />
<documentCache class="solr.FastLRUCache" size="10240" initialSize="4096"
autowarmCount="0" />
<cache name="perSegFilter" class="solr.search.LRUCache" size
="10" initialSize="0" autowarmCount="10" regenerator="solr.NoOpRegenerator"
/>
Not sure what this means...
On Sat, Dec 12, 2015 at 1:13 AM, Vincenzo D'Amore <v.dam...@gmail.com>
wrote:
> Thanks Erick, Mark,
>
> I'll raise maxTime asap.
> Just to be sure understand, given that I have openSearcher=false, I
> suppose it shouldn't trigger autowarming at least until a commit is
> executed, shouldn't it?
>
> Anyway, I don't understand, given that maxTime is very aggressive, why
> hard commit takes so long.
>
> Thanks again for your answers.
> Vincenzo
>
>
> On Fri, Dec 11, 2015 at 7:22 PM, Erick Erickson <erickerick...@gmail.com>
> wrote:
>
>> First of all, your autocommit settings are _very_ aggressive. Committing
>> every second is far to frequent IMO.
>>
>> As an aside, I generally prefer to omit the maxDocs as it's not all
>> that predictable,
>> but that's a personal preference and really doesn't bear on your problem..
>>
>> My _guess_ is that you are doing a lot of autowarming. The number of docs
>> doesn't really matter if your autowarming is taking forever, your Solr
>> logs
>> should report the autowarm times at INFO level, have you checked those?
>>
>> The commit settings shouldn't be a problem in terms of your server dying,
>> the indexing process flushes docs to the tlog independent of committing so
>> upon restart they should be recovered. Here's a blog on the subject:
>>
>>
>> https://lucidworks.com/blog/2013/08/23/understanding-transaction-logs-softcommit-and-commit-in-sorlcloud/
>>
>> Best,
>> Erick
>>
>> On Fri, Dec 11, 2015 at 8:24 AM, Vincenzo D'Amore <v.dam...@gmail.com>
>> wrote:
>> > Hi all,
>> >
>> > I have a SolrCloud cluster with a collection (2.5M docs) with 3 shards
>> and
>> > 15 replicas.
>> > There is a solrj application that feeds the collection, updating few
>> > documents every hour, I don't understand why, at end of process, the
>> hard
>> > commit takes about 8/10 minutes.
>> >
>> > Even if there are only few hundreds of documents.
>> >
>> > This is the autocommit configuration:
>> >
>> > <autoCommit>
>> > <maxDocs>10000</maxDocs>
>> > <maxTime>1000</maxTime>
>> > <openSearcher>false</openSearcher>
>> > </autoCommit>
>> >
>> > In your experience why hard commit takes so long even for so few
>> documents?
>> >
>> > Now I'm changing the code to softcommit, calling commit (waitFlush =
>> > false, waitSearcher
>> > = false, softCommit = true);
>> >
>> > solrServer.commit(false, false, true);.
>> >
>> > I have configured NRTCachingDirectoryFactory, but I'm a little bit
>> worried
>> > if a server goes down (something like: kill -9, SolrCloud crashes, out
>> of
>> > memory, etc.), and if, using this strategy
>> softcommit+NRTCachingDirectory,
>> > SolrCloud instance could not recover a replica.
>> >
>> > Should I worry about this new configuration? I was thinking to take a
>> > snapshot of everything every day, in order to recover immediately the
>> > index. Could this be considered a best practice?
>> >
>> > Thanks in advance for your time,
>> > Vincenzo
>> >
>> > --
>> > Vincenzo D'Amore
>> > email: v.dam...@gmail.com
>> > skype: free.dev
>> > mobile: +39 349 8513251
>>
>
>
>
> --
> Vincenzo D'Amore
> email: v.dam...@gmail.com
> skype: free.dev
> mobile: +39 349 8513251
>
--
Vincenzo D'Amore
email: v.dam...@gmail.com
skype: free.dev
mobile: +39 349 8513251
