Hi Chris, Thank you very much for your response! It was very instructive. I knew some performance tips to improve search and I configured a very low merge factor (<mergeFactor>2</mergeFactor>) to boost search operations instead of indexation ones. I haven't got a deep knowledge of internal Lucene behavior in this case, but I thought that somehow an optimization operation may rebuild the index checking and fixing corrupted segments, merging again whatever it should merge, etc., and finally the "new" master index will be a better index where to insert new data frequently.
One last question: do you think that this kind of scenario where I continuously index and replicate data will corrupt the index? In the past I developed a simple tool using a Lucene class to check the index and alert me if it's corrupted or not, so if you think that this scenario is dangerous maybe I can reuse that tool to prevent weird production situations. Best, - Luis Cappa 2014-02-05 Chris Hostetter <hossman_luc...@fucit.org>: > > : I've got an scenario where I index very frequently on master servers and > : replicate to slave servers with one minute polling. Master indexes are > : growing fast and I would like to optimize indexes to improve search > : queries. However... > > For a scenerio where your index is changing that rapidly, you don't wnat > to use the optimize command at all -- it's not going to improve the > performance of anything... > > > https://wiki.apache.org/solr/SolrPerformanceFactors#Optimization_Considerations > > You may want to optimize an index in certain situations -- ie: if you > build your index once, and then never modify it. > > If you have a rapidly changing index, rather than optimizing, you likely > simply want to use a lower merge factor. Optimizing is very expensive, and > if the index is constantly changing, the slight performance boost will not > last long. The tradeoff is not often worth it for a non static index. > > In a master slave setup, sometimes you may also want to optimize on the > master so that slaves serve from a single segment index. This will can > greatly increase the time to replicate the index though, so this is often > not desirable either. > > > > -Hoss > http://www.lucidworks.com/ > -- - Luis Cappa
