The pattern I’ve seen used looks like this:
User application (e.g. browser) >> Global load balancer >> Service
instances (e.g. tomcat) >> Geode cluster
If you have the Geode clusters connected via WAN, you can redirect traffic to
different data centers by tweaking the LB config.
Anthony
> On Apr 16, 2019, at 2:58 AM, aashish choudhary <[email protected]>
> wrote:
>
> So with WAN we will be active/active all time. I agree hardest is to figure
> out when data centers are actually down. We are evaluating multiple
> approaches as of now.
>
> On that note would you recommend(possibility any since connection is mostly
> tcp/ip) using some load balancer NGINX or something to handle data center
> failure.
>
> With best regards,
> Ashish
>
> On Tue, Apr 16, 2019, 12:54 AM Michael Stolz <[email protected]
> <mailto:[email protected]>> wrote:
> We have come across these kinds of use-cases.
> The hardest part is figuring out that one of the data centers is ACTUALLY
> down.
>
> If you can work out a way to be active/active at all times and guard against
> update collisions by using data structures that protect themselves (e.g.
> CRDTs) that would make the whole thing a lot easier.
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771
>
>
>
> On Mon, Apr 15, 2019 at 1:19 PM aashish choudhary
> <[email protected] <mailto:[email protected]>> wrote:
> Thanks Mike. Our use case is heavily reliant on Geode(no fallback to
> database) and business expectation is that there will be no downtime to
> consumer application because of complete failure on one data center. Which
>
> Have you came across such cases with Geode/Gemfire?
>
> Regarding catching those exceptions and making a switch I agree with you that
> it would be tricky to make switch as you explained.
>
> Even with rolling restart there will be a downtime and some manual steps will
> be required to accomplish that.
>
> With best regards,
> Ashish
>
> On Thu, Apr 11, 2019, 10:18 PM Michael Stolz <[email protected]
> <mailto:[email protected]>> wrote:
> Yes you can catch the exceptions for no locators available and no servers
> available.
> You will probably want to wait for a period of time after first seeing this,
> because the cluster might be restarting and will be back in just a minute or
> so.
>
> The switch-over can be tricky without just restarting your client.
>
> All saved references to everything having to do with ClientCache, Cache,
> Region, or anything else that communicates need to be forgotten and
> re-established.
> This can be particularly challenging if you are using a framework that might
> remember some of this stuff on your behalf.
> I have usually recommended rolling restart of the clients with the new
> locator addresses because it is sure to work and not have any hidden issues
> with calls in progress or subscriptions or anything like that.
>
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771
>
>
>
> On Thu, Apr 11, 2019 at 9:31 AM aashish choudhary
> <[email protected] <mailto:[email protected]>> wrote:
> Thanks Mike.
>
> Yes we are using wan replication. We want the switch to be an automatic step.
> As soon as prod cluster fails we need to switch to cob without any restart of
> the client application.
>
> One way we are thinking of is probably catching those locator not available
> sort of exception and then make a switch. Any thoughts?
>
>
> With best regards,
> Ashish
>
>
> On Thu, Apr 11, 2019, 1:42 AM Michael Stolz <[email protected]
> <mailto:[email protected]>> wrote:
> If the data centers are far apart you will want to use the bi-directional
> GemFire WAN Gateway to replicate between clusters.
>
> The trickiest part is figuring out when to switch. If you already have a
> mechanism for that then that's great.
>
> Once you know for sure you want to switch, the easiest way is to install a
> gemfire.properties file on the client machines that points to the locators in
> the other data center and restart the clients.
>
> There is a programmatic way to do it but is a lot more code and work than
> this way.
>
> Feel free to ask any additional questions here.
>
>
>
> --
> Mike Stolz
> Principal Engineer, GemFire Product Lead
> Mobile: +1-631-835-4771
>
> On Wed, Apr 10, 2019, 2:01 PM aashish choudhary <[email protected]
> <mailto:[email protected]>> wrote:
> Hi,
>
> We have a scenario where in we need to switch over to a different data center
> automatically when any failure occurs in the existing cluster.
>
> Any recommendations?
>
>
> With best regards,
> Ashish
