On 05.03.2018 11:07, azurIt wrote:
>> Hi,
>>
>> we have activated fts-solr about a week ago and immediately started to
>> experience really *low* performance with MOVE and EXPUNGE commands.
>> After several days of googling, tcpdumping and straceing i was able to
>> find and resolve the problem.
>>
>> We are using Dovecot 2.2.27 from Debian Jessie (jessie-backports),
>> which is doing a soft commit in solr after every MOVE or EXPUNGE
>> command - this behavior cannot be, currently, changed. The problem is
>> that this was causing every MOVE/EXPUNGE to take about 6 seconds to
>> complete. The problem appears to be in very old version of Solr -
>> 3.6.2 (!!). This is the only version which is shipped with current
>> (Jessie) and also next (Stretch) version of Debian, don't ask my why,
>> i don't understand it either. Solr versions below 4.0 are NOT
>> supporting soft commits, so all commits are hard and this was the
>> problem. Finally, i decided to patch our Dovecot to not send a commit
>> at all and everything started to be super fast. I'm doing hard commits
>> every minute via cron so the only consequence of this is that you
>> cannot search for messages delivered before less then a minute (which
>> you, usually, don't need to do anyway).
>>
>> While googling i also find out that Solr supports autoCommit function
>> (and from version 4.0 also autoSoftCommit), so there's no reason for
>> Dovecot to handle this on it's own (and potentially doing hundreds or
>> thousands of soft commits every second) - you can just set Solr to,
>> for example, do autoSoftCommit every second and autoCommit every minute:
>> https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConfig#UpdateHandlersinSolrConfig-autoCommit
>>
>> Also this wiki page should be updated with warning about old versoins
>> of Solr not supporting soft commits (you could also mention the
>> auto[Soft]Commit function):
>> http://wiki2.dovecot.org/Plugins/FTS/Solr
>>
>> I suggest to allow completely disable Solr commits in Dovecot by
>> configuration, so people like me can handle this easily. What do you
>> think?
>>
>> azur
>
>
>
> Hi,
>
> any news on this? Even Solr documentation suggests to NOT doing commits from
> applications:
> https://lucene.apache.org/solr/guide/6_6/shards-and-indexing-data-in-solrcloud.html#ShardsandIndexingDatainSolrCloud-IgnoringCommitsfromClientApplicationsinSolrCloud
>
> Thanks for not ignoring me.
>
> azur
You are not being ignored. We'll attend to this eventually.
Aki
