Thank you everyone for showing interest and sharing your thoughts. The overall approach I've been thinking about involves: 1. A consumer app that reads from a (local) message queue and writes to a local Solr instance. 2. Update Request Processor in Solr to forward the updates to an outgoing local queue.
The messaging system in the middle will be responsible to handle the replication from the local source queue to remote destination queues that are local to the remote clusters. This will allow the individual clusters to be agnostic of the remote cluster location. I'll need a few more days before I share the document, but just wanted to give an idea about the data flow we've been using for a few years. As expected, there are a few caveats and restrictions with this approach which I'll include with the document :) -Anshum On Sun, Dec 6, 2020 at 1:53 PM Bram Van Dam <bram.van...@intix.eu> wrote: > We've had some experience with this. As per usual, customers tend to > want the holy grail: no data loss when one data centre blows up, but no > increased latency when updating data. This then somehow, magically, has > to work over a slow uplink between two data centres without saturating > the link. > > Currently we use NRT replicas across data centres. Which does add some > latency, but consistency is a bit more important for us. Overall, this > works pretty well. > > The biggest problems we've experienced have all been related to > recovering replicas across a slow data centre uplink. A saturated link > can cause multiple replicas to lag behind, and when this gets too bad, > they too will go into recovery, and then shit really hits the fan. > > I'm not sure whether there are any easy ways of improving that > behaviour. Limiting max bandwidth per solr instance during recovery? > Slow recovery is better than destructive recovery. > > External tools like Kafka add a lot of operational overhead. One of the > great things about SolrCloud is how simple the whole replication setup is. > > > > On 06/12/2020 14:46, Erick Erickson wrote: > >> I can see at least two different approaches here, your mention of SolrJ > seems to hint at the first one: > >> 1. Get the data as it comes from the client and fork it to local and > remote data centers, > >> 2. Create (an asynchronous) stream replicating local data center data > to remote. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Anshum Gupta
