Re: Optimizing index on Slave

2014-01-23 Thread Michael Della Bitta
I'm not really aware enough of the Solr/Lucene internals to tell you whether that's possible or not. One thing occurred to me: What happens if you take optimize out of the replication triggers in the replication handler? str name=replicateAfteroptimize/str Michael Della Bitta Applications

Re: Optimizing index on Slave

2014-01-22 Thread Salman Akram
We do. We have a lot of updates/deletes every day and a weekly optimization definitely gives a considerable improvement so don't see a downside to it except the complete replication part which is not an issue on local network.

Re: Optimizing index on Slave

2014-01-22 Thread Michael Della Bitta
Salman, To my knowledge, there's not a great way of doing this. Perhaps if your dataset were based on a time series, you could shard by date, and then only a smaller segment of your data would be updated and therefore need to be sent each week? Michael Della Bitta Applications Developer o:

Re: Optimizing index on Slave

2014-01-22 Thread Salman Akram
Unfortunately we can't do sharding right now. If we optimize on master and slave separately the file names and sizes are same. I think it's just the version no that is different. Maybe if there was a to copy master version to slave that would resolve this issue?

Re: Optimizing index on Slave

2014-01-21 Thread Michael Della Bitta
Taking a step back: Are you sure you need to optimize? It might be hurting you more than it helps. Michael Della Bitta Applications Developer o: +1 646 532 3062 appinions inc. “The Science of Influence Marketing” 18 East 41st Street New York, NY 10017 t: @appinions

Optimizing index on Slave

2014-01-20 Thread Salman Akram
All, I know normally index should be optimized on master and it will be replicated to slave but we have an issue with the network bandwidth. We optimize indexes weekly (total size is around 1.5TB). We have few slaves set up on local network so replication the whole indexes is not a big issue.