Hi,
----- Original Message ---- > > > Oh, there is no DB involved. Think of a document stream continuously > > coming >in, > > a component listening to that stream, grabbing docs, and pushing it to > > master(s). > > I don't think Solr is designed for this use case, eg, I wouldn't > expect deterministic results with the current architecture as it's > something that's inherently a a key component of [No]SQL databases. You mean it's not possible to have 2 masters that are in nearly real-time sync? How about with DRBD? I know people use DRBD to keep 2 Hadoop NNs (their edit logs) in sync to avoid the current NN SPOF, for example, so I'm thinking this could be doable with Solr masters, too, no? Otis ---- Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ > On Wed, Mar 9, 2011 at 8:49 AM, Otis Gospodnetic > <otis_gospodne...@yahoo.com> wrote: > > Hi, > > > > ----- Original Message ---- > > > >> If you're using the delta import handler the problem would seem to go > >> away because you can have two separate masters running at all times, > >> and if one fails, you can then point the slaves to the secondary > >> master, that is guaranteed to be in sync because it's been importing > >> from the same database? > > > > Oh, there is no DB involved. Think of a document stream continuously > > coming >in, > > a component listening to that stream, grabbing docs, and pushing it to > > master(s). > > > > Otis > > > > > > > >> On Tue, Mar 8, 2011 at 8:45 PM, Otis Gospodnetic > >> <otis_gospodne...@yahoo.com> wrote: > >> > Hello, > >> > > >> > What are some common or good ways to handle indexing (master) fail-over? > >> > Imagine you have a continuous stream of incoming documents that you > >> > have >to > >> > index without losing any of them (or with losing as few of them as > >>possible). > >> > How do you set up you masters? > >> > In other words, you can't just have 2 masters where the secondary is the > >> > Repeater (or Slave) of the primary master and replicates the index > >>periodically: > >> > you need to have 2 masters that are in sync at all times! > >> > How do you achieve that? > >> > > >> > * Do you just put N masters behind a LB VIP, configure them both to >point to > >>the > >> > index on some shared storage (e.g. SAN), and count on the LB to >fail-over to > >>the > >> > secondary master when the primary becomes unreachable? > >> > If so, how do you deal with index locks? You use the Native lock and >count > >>on > >> > it disappearing when the primary master goes down? That means you > >> > count >on > >>the > >> > whole JVM process dying, which may not be the case... > >> > > >> > * Or do you use tools like DRBD, Corosync, Pacemaker, etc. to keep 2 > > masters > >> > with 2 separate indices in sync, while making sure you write to only 1 > >> > >of > >>them > >> > via LB VIP or otherwise? > >> > > >> > * Or ... > >> > > >> > > >> > This thread is on a similar topic, but is inconclusive: > >> > http://search-lucene.com/m/aOsyN15f1qd1 > >> > > >> > Here is another similar thread, but this one doesn't cover how 2 >masters > > are > >> > kept in sync at all times: > >> > http://search-lucene.com/m/aOsyN15f1qd1 > >> > > >> > Thanks, > >> > Otis > >> > ---- > >> > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > >> > Lucene ecosystem search :: http://search-lucene.com/ > >> > > >> > > >> > > >
