On 1/24/2014 10:36 AM, sureshrk19 wrote:
I'm not committing each document but, have following configuration in solrconfig.xml (commit every 5mins).<autoCommit> <maxTime>300000</maxTime> <openSearcher>false</openSearcher> </autoCommit> Also, if you look at my master config, I do not have 'optimize'. <str name="replicateAfter">startup</str> <str name="replicateAfter">commit</str> Is there any way other option which triggers 'optimize'?
I think Erick was actually asking if you are optimizing your index frequently, not whether you have replication configured to replicate after optimize.
Optimizing your index (a forced merge down to one Lucene index segment) is something you have to do yourself. It won't happen automatically. If you optimize your index, all old segments are gone and only a single new segment remains. Even if you don't replicate immediately, the next time you commit, the entire index will need to be copied to the slave.
Your autoCommit cannot be the only committing that you do, because that configuration will not make new documents visible - it has openSearcher=false. Therefore if you are adding new content, you must be doing additional soft commits, or hard commits with openSearcher=true. This might be accomplished with a parameter on your updates, like commit, softCommit, or commitWithin. It might also be an explicit commit.
Optimizing *IS* a useful feature, but if you optimize very frequently (especially if it's done every time you add new documents), Solr's performance will really suffer.
Personal anecdote: One of my shards is very tiny and holds all new content. That gets optimized once an hour. In general, this is pretty frequently, but it happens very quickly, so in my setup it's not excessive. That is a LOT more often than what I do for my other shards, the large ones. I optimize one of those once every day, so each one only gets optimized once every six days.
Thanks, Shawn
