Hello Ashish, Hope all is well.
FYI - here is a spring based example; https://github.com/vmwarepivotallabs/dataTx-geode-circuitbreaker-demo > On Apr 16, 2020, at 6:48 AM, aashish choudhary <[email protected]> > wrote: > > > I think i don't need to create ClientCache again in the catch block as i am > using separate pools for 2 clusters. Updated the code. > https://github.com/userac/GeodeExample/blob/master/src/main/java/Example.java > With Best Regards, > Ashish > > >> On Thu, Apr 16, 2020 at 3:01 PM aashish choudhary >> <[email protected]> wrote: >> Here is the simple code example that i tried for client side failover. For >> my use case i will always try to connect to primary cluster first and if it >> fails it should failover to secondary. I understand it may slow things as i >> will be creating a connection on each request but there will be no issue in >> getting response. Not sure if there is any better way to do this. I should >> probably add something to stop bouncing back and forth if both clusters are >> down. >> >> https://github.com/userac/GeodeExample/blob/master/src/main/java/Example.java >> >> >> Let me know what you think. >> >> With Best Regards, >> Ashish >> >> >>> On Wed, Apr 15, 2020 at 4:32 PM aashish choudhary >>> <[email protected]> wrote: >>> Replying to old thread. >>> Currently I am trying test client side failover programtically. Are there >>> any working examples /best practices for this? >>> >>> Idea is to failover to different secondary cluster if the primary fails. I >>> will try to share what i am able to do so far. >>> >>> But if anyone has a working example for this then it would be great. >>> >>> With best regards, >>> Ashish >>> >>>> On Tue, Apr 16, 2019, 9:37 PM Anthony Baker <[email protected]> wrote: >>>> The pattern I’ve seen used looks like this: >>>> >>>> User application (e.g. browser) >> Global load balancer >> Service >>>> instances (e.g. tomcat) >> Geode cluster >>>> >>>> If you have the Geode clusters connected via WAN, you can redirect traffic >>>> to different data centers by tweaking the LB config. >>>> >>>> >>>> Anthony >>>> >>>> >>>>> On Apr 16, 2019, at 2:58 AM, aashish choudhary >>>>> <[email protected]> wrote: >>>>> >>>>> So with WAN we will be active/active all time. I agree hardest is to >>>>> figure out when data centers are actually down. We are evaluating >>>>> multiple approaches as of now. >>>>> >>>>> On that note would you recommend(possibility any since connection is >>>>> mostly tcp/ip) using some load balancer NGINX or something to handle data >>>>> center failure. >>>>> >>>>> With best regards, >>>>> Ashish >>>>> >>>>>> On Tue, Apr 16, 2019, 12:54 AM Michael Stolz <[email protected]> wrote: >>>>>> We have come across these kinds of use-cases. >>>>>> The hardest part is figuring out that one of the data centers is >>>>>> ACTUALLY down. >>>>>> >>>>>> If you can work out a way to be active/active at all times and guard >>>>>> against update collisions by using data structures that protect >>>>>> themselves (e.g. CRDTs) that would make the whole thing a lot easier. >>>>>> -- >>>>>> Mike Stolz >>>>>> Principal Engineer, GemFire Product Lead >>>>>> Mobile: +1-631-835-4771 >>>>>> >>>>>> >>>>>> >>>>>>> On Mon, Apr 15, 2019 at 1:19 PM aashish choudhary >>>>>>> <[email protected]> wrote: >>>>>>> Thanks Mike. Our use case is heavily reliant on Geode(no fallback to >>>>>>> database) and business expectation is that there will be no downtime to >>>>>>> consumer application because of complete failure on one data center. >>>>>>> Which >>>>>>> >>>>>>> Have you came across such cases with Geode/Gemfire? >>>>>>> >>>>>>> Regarding catching those exceptions and making a switch I agree with >>>>>>> you that it would be tricky to make switch as you explained. >>>>>>> >>>>>>> Even with rolling restart there will be a downtime and some manual >>>>>>> steps will be required to accomplish that. >>>>>>> >>>>>>> With best regards, >>>>>>> Ashish >>>>>>> >>>>>>>> On Thu, Apr 11, 2019, 10:18 PM Michael Stolz <[email protected]> wrote: >>>>>>>> Yes you can catch the exceptions for no locators available and no >>>>>>>> servers available. >>>>>>>> You will probably want to wait for a period of time after first seeing >>>>>>>> this, because the cluster might be restarting and will be back in just >>>>>>>> a minute or so. >>>>>>>> >>>>>>>> The switch-over can be tricky without just restarting your client. >>>>>>>> >>>>>>>> All saved references to everything having to do with ClientCache, >>>>>>>> Cache, Region, or anything else that communicates need to be forgotten >>>>>>>> and re-established. >>>>>>>> This can be particularly challenging if you are using a framework that >>>>>>>> might remember some of this stuff on your behalf. >>>>>>>> I have usually recommended rolling restart of the clients with the new >>>>>>>> locator addresses because it is sure to work and not have any hidden >>>>>>>> issues with calls in progress or subscriptions or anything like that. >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Mike Stolz >>>>>>>> Principal Engineer, GemFire Product Lead >>>>>>>> Mobile: +1-631-835-4771 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Thu, Apr 11, 2019 at 9:31 AM aashish choudhary >>>>>>>>> <[email protected]> wrote: >>>>>>>>> Thanks Mike. >>>>>>>>> >>>>>>>>> Yes we are using wan replication. We want the switch to be an >>>>>>>>> automatic step. As soon as prod cluster fails we need to switch to >>>>>>>>> cob without any restart of the client application. >>>>>>>>> >>>>>>>>> One way we are thinking of is probably catching those locator not >>>>>>>>> available sort of exception and then make a switch. Any thoughts? >>>>>>>>> >>>>>>>>> >>>>>>>>> With best regards, >>>>>>>>> Ashish >>>>>>>>> >>>>>>>>> >>>>>>>>>> On Thu, Apr 11, 2019, 1:42 AM Michael Stolz <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> If the data centers are far apart you will want to use the >>>>>>>>>> bi-directional GemFire WAN Gateway to replicate between clusters. >>>>>>>>>> >>>>>>>>>> The trickiest part is figuring out when to switch. If you already >>>>>>>>>> have a mechanism for that then that's great. >>>>>>>>>> >>>>>>>>>> Once you know for sure you want to switch, the easiest way is to >>>>>>>>>> install a gemfire.properties file on the client machines that points >>>>>>>>>> to the locators in the other data center and restart the clients. >>>>>>>>>> >>>>>>>>>> There is a programmatic way to do it but is a lot more code and work >>>>>>>>>> than this way. >>>>>>>>>> >>>>>>>>>> Feel free to ask any additional questions here. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Mike Stolz >>>>>>>>>> Principal Engineer, GemFire Product Lead >>>>>>>>>> Mobile: +1-631-835-4771 >>>>>>>>>> >>>>>>>>>>> On Wed, Apr 10, 2019, 2:01 PM aashish choudhary >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> We have a scenario where in we need to switch over to a different >>>>>>>>>>> data center automatically when any failure occurs in the existing >>>>>>>>>>> cluster. >>>>>>>>>>> >>>>>>>>>>> Any recommendations? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> With best regards, >>>>>>>>>>> Ashish >>>>
