Not sure if KEA has a way to deal with it "gracefully", but in general, once leases are being given out in split-brain, the "train has left the station". A few things that can help: - ping-verification on lease helps avoid conflicts (although kea doesn't do it https://kb.isc.org/docs/why-doesnt-kea-support-ping-check ) - figure out which of your nodes has the most leases, and use that as your source of truth. - If they have the same amount of leases, use the one with the NEWEST leases as truth, since clients with older leases will expire and re-discover on their own earlier. - Once they're back in sync, force reboots if you can (yeah, this is pretty impactful). - lower your leases while you're trying to get things back in sync. It causes more traffic, but helps things heal on their own.
Dan >-----Original Message----- >From: Kea-users <[email protected]> On Behalf Of perl-list >Sent: Wednesday, February 9, 2022 8:00 AM >To: kea-users <[email protected]> >Subject: [EXTERNAL] Re: [Kea-users] recover from split brain? > >I think he was asking what happens if the servers can't see each other and >both start answering clients, possibly creating IP conflicts that must be >resolved at some point. Imagine a split fiber ring where the servers can't see >each other and the clients on each half of the ring can't see each other. If >this >persists for a while, duplicate IPs could be allocated. ISC DHCP solved this >problem by not allocating addresses that were assigned to the other server >unless you hand set "partner down" mode meaning you knew the other >server was down and duplicate IPs wouldn't be assigned. > >----- Original Message ----- >> From: "Chad Catlett" <[email protected]> >> To: "David Ramsey" <[email protected]>, "kea-users" >> <[email protected]> >> Sent: Tuesday, February 8, 2022 6:21:53 PM >> Subject: Re: [Kea-users] recover from split brain? > >> On 2/8/22 14:12, David Ramsey wrote: >> > Lab testing I've done shows that the Active/Standby HA configuration >> > works well if one or the other DHCP servers fails completely. >> > Automatic takeover and recovery work nicely. > >> > If an Active/Standby HA implementation were to go "split brain" with >> > core network severed between DHCP servers, however, both would go >> > active and the lease DBs would no longer be in sync. > >> > How to recover from that when the network is restored? I know there >> > are a ton of variables involved - lease times, how long network was >> > segmented, # subscribers, etc.... but I am wrestling with what would >> > be the optimal approach/logic to consolidate and recover ? > >> > Help? > >> > Thanks folks, --David > > > > > >> The admin docs[0] do a fairly decent job of explaining how HA failover >> situations are recovered from. The amount of time it takes is directly >> related to the dhcp statemachine timers, which are influenced by your >> settings. > >> 0: >> https://urldefense.com/v3/__https://kea.readthedocs.io/en/kea-2.1.2/ar >> m/hooks.html*scope-transition-in-a-partner-down-case__;Iw!!Hit2Ag!i5gg >> 7MxOAanUKgfx1-ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzE7lv1FOA$ > >> Chad > >> -- >> Chad Catlett >> [email protected] > >> -- >> ISC funds the development of this software with paid support subscriptions. >> Contact us at >https://urldefense.com/v3/__https://www.isc.org/contact/__;!!Hit2Ag!i5gg7 >MxOAanUKgfx1-ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzH2hVg9Ng$ for >more information. > >> To unsubscribe visit >https://urldefense.com/v3/__https://lists.isc.org/mailman/listinfo/kea- >users__;!!Hit2Ag!i5gg7MxOAanUKgfx1- >ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzHKeLFyhw$ . > >> Kea-users mailing list >> [email protected] >> https://urldefense.com/v3/__https://lists.isc.org/mailman/listinfo/kea >> -users__;!!Hit2Ag!i5gg7MxOAanUKgfx1- >ZohtyrYmmud5yJy217j4ao8kUVcctxXvur >> hzHKeLFyhw$ >-- >ISC funds the development of this software with paid support subscriptions. >Contact us at >https://urldefense.com/v3/__https://www.isc.org/contact/__;!!Hit2Ag!i5gg7 >MxOAanUKgfx1-ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzH2hVg9Ng$ for >more information. > >To unsubscribe visit >https://urldefense.com/v3/__https://lists.isc.org/mailman/listinfo/kea- >users__;!!Hit2Ag!i5gg7MxOAanUKgfx1- >ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzHKeLFyhw$ . > >Kea-users mailing list >[email protected] >https://urldefense.com/v3/__https://lists.isc.org/mailman/listinfo/kea- >users__;!!Hit2Ag!i5gg7MxOAanUKgfx1- >ZohtyrYmmud5yJy217j4ao8kUVcctxXvurhzHKeLFyhw$ -- ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information. To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users. Kea-users mailing list [email protected] https://lists.isc.org/mailman/listinfo/kea-users
