Mark Miller <markrmil...@gmail.com> wrote on 11/12/2009 07:18:03 PM: > Ah, the pains of optimization. Its kind of just how it is. One solution > is to use two boxes and replication - optimize on the master, and then > queries only hit the slave. Out of reach for some though, and adds many > complications.
Yes, in my use case 2 boxes isn't a great option. > Another kind of option is to use the partial optimize feature: > > <optimize maxOptimizeSegments="5"/> > > Using this, you can optimize down to n segments and take a shorter hit > each time. Is this a 1.4 feature? I'm planning to migrate to 1.4, but it'll take a while since I have to port custom code forward, including a query parser. > Also, if optimizing is so painful, you might lower the merge factor > amortize that pain better. Thats another way to slowly get there - if > you lower the merge factor, as merging takes place, the new merge factor > will be respected, and semgents will merge down. A merge factor of 2 > (the lowest) will make it so you only ever have 2 segments. Sometimes > that works reasonably well - you could try 3-6 or something as well. > Then when you do your partial optimizes (and eventually a full optimize > perhaps), you want have so far to go. So this will slow down indexing but speed up optimize somewhat? Unfortunately right now I lose docs I'm indexing, as well slowing searching to a crawl. Ugh. I've got plenty of CPU horsepower. This is where having the ability to optimize on another filesystem would be useful. Would it perhaps make sense to set up a master/slave on the same machine? Then I suppose I can have an index being optimized that might not clobber the search. Would new indexed items still be dropped on the floor? Thanks, Jerry
