Hi,


----- Original Message ----
>
> I'd honestly think about buffer the incoming documents in some store that's  
>actually made for fail-over persistence reliability, maybe CouchDB or 
>something.  
>And then that's taking care of not losing anything, and the problem becomes 
>how  
>we make sure that our solr master indexes are kept in sync with the actual  
>persistent store; which I'm still not sure about, but I'm thinking it's a  
>simpler problem. The right tool for the right job, that kind of failover  
>persistence is not solr's specialty. 
>

But check this!  In some cases one is not allowed to save content to disk 
(think 
copyrights).  I'm not making this up - we actually have a customer with this 
"cannot save to disk" (but can index) requirement.

So buffering to disk is not an option, and buffering in memory is not practical 
because of the input document rate and their size.

Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Lucene ecosystem search :: http://search-lucene.com/



> From: Otis Gospodnetic [otis_gospodne...@yahoo.com]
> Sent:  Tuesday, March 08, 2011 11:45 PM
> To: solr-user@lucene.apache.org
> Subject:  True master-master fail-over without data gaps
> 
> Hello,
> 
> What are  some common or good ways to handle indexing (master) fail-over?
> Imagine you  have a continuous stream of incoming documents that you have to
> index without  losing any of them (or with losing as few of them as possible).
> How do you  set up you masters?
> In other words, you can't just have 2 masters where the  secondary is the
> Repeater (or Slave) of the primary master and replicates the  index 
>periodically:
> you need to have 2 masters that are in sync at all  times!
> How do you achieve that?
> 
> * Do you just put N masters behind a  LB VIP, configure them both to point to 
>the
> index on some shared storage  (e.g. SAN), and count on the LB to fail-over to 
>the
> secondary master when the  primary becomes unreachable?
> If so, how do you deal with index locks?   You use the Native lock and count 
on
> it disappearing when the primary master  goes down?  That means you count on 
>the
> whole JVM process dying, which  may not be the case...
> 
> * Or do you use tools like DRBD, Corosync,  Pacemaker, etc. to keep 2 masters
> with 2 separate indices in sync, while  making sure you write to only 1 of 
them
> via LB VIP or otherwise?
> 
> * Or  ...
> 
> 
> This thread is on a similar topic, but is inconclusive:
>   http://search-lucene.com/m/aOsyN15f1qd1
> 
> Here is another similar  thread, but this one doesn't cover how 2 masters are
> kept in sync at all  times:
>   http://search-lucene.com/m/aOsyN15f1qd1
> 
> Thanks,
> Otis
> ----
> Sematext  :: http://sematext.com/ ::  Solr - Lucene - Nutch
> Lucene ecosystem search :: http://search-lucene.com/
> 
> 

Reply via email to