Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread lame
Hi guys, I have master slave replication enabled. Slave is replicating every 3 minutes and I encourage problems while I'm performing full import command on master (which takes about 7 minutes). Slave repliacates partial index about 200k documents out of 700k. After next repliacation full index is

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
Do you commit to often? Slaves won't replicate if while master is indexing if you don't send commits. Can you only commit once the indexing finishes? On Monday 14 March 2011 14:04:51 lame wrote: Hi guys, I have master slave replication enabled. Slave is replicating every 3 minutes and I

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread lame
I don't commit at all we use Dataimporter, but I have a feeling that it could be done by DIH (autocommit is it possible)? 2011/3/14 Markus Jelsma markus.jel...@openindex.io: Do you commit to often? Slaves won't replicate if while master is indexing if you don't send commits. Can you only

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
In solrconfig there might be a autocommit section enabled. On Monday 14 March 2011 14:18:42 lame wrote: I don't commit at all we use Dataimporter, but I have a feeling that it could be done by DIH (autocommit is it possible)? 2011/3/14 Markus Jelsma markus.jel...@openindex.io: Do you

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread lame
It looks like (we don't have autocommit section in solr.DirectUpdateHandler2, is ramBufferSizeMB is responsible for that?): indexDefaults useCompoundFilefalse/useCompoundFile mergeFactor10/mergeFactor ramBufferSizeMB320/ramBufferSizeMB maxMergeDocs2147483647/maxMergeDocs

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
These settings don't affect a commit. But, the maxPendingDeletes might but i'm unsure. If you commit on the master and slaves are configured to replicate on commit, it all should have the same index version. On Monday 14 March 2011 14:42:51 lame wrote: It looks like (we don't have autocommit

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread lame
We have also commits from application (besides full import) - maybe that is the case. If you don't have any other ideas I'll probably try reindexing second core, than swap cores and run delta import (to import documets added in the meantime). 2011/3/14 Markus Jelsma markus.jel...@openindex.io:

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Markus Jelsma
Yes, commits from the application will interfere indeed. If your business scenario allows for using always optimized indices you might choose to only replicate on optimize. On Monday 14 March 2011 18:45:15 lame wrote: We have also commits from application (besides full import) - maybe that is

Re: Solr 1.4 replication - partial index on slave while indexing master

2011-03-14 Thread Bill Bell
Turn off all autocommitting.. On 3/14/11 7:04 AM, lame l...@o2.pl wrote: Hi guys, I have master slave replication enabled. Slave is replicating every 3 minutes and I encourage problems while I'm performing full import command on master (which takes about 7 minutes). Slave repliacates partial