On Fri, Feb 10, 2017 at 12:27 PM, Anna Taraday <akamyshnik...@mirantis.com> wrote: > Hello everyone! > > In Juno in Neutron was implemented L3 HA feature based on Keepalived (VRRP). > During next cycles it was improved, we performed scale testing [1] to find > weak places and tried to fix them. The only alternative for L3 HA with VRRP > is router rescheduling performed by Neutron server, but it is significantly > slower and depends on control plane. > > What issues we experienced with L3 HA VRRP? > > Bugs in Keepalived (bad versions) [2] > Split brain [3] > Complex structure (ha networks, ha interfaces) - which actually cause races > that we were fixing during Liberty, Mitaka and Newton. > > This all is not critical, but this is a bad experience and not everyone > ready (or want) to use Keepalived approach. > > I think we can make things more flexible. For example, we can allow user to > use external services like etcd instead of Keepalived to synchronize current > HA state across agents. I've done several experiments and I've got failover > time comparable to L3 HA with VRRP. Tooz [4] can be used to abstract from > concrete backend. For example, it can allow us to use Zookeeper, Redis and > other backends to store HA state. > > What I want to propose? > > I want to bring up idea that Neutron should have some general classes for L3 > HA which will allow to use not only Keepalived but also other backends for > HA state. This at least will make it easier to try some other approaches and > compare them with existing ones. > > Does this sound reasonable?
I understand that the intention is to add pluggability upstream so that you could examine the viability of alternative solutions. I'd advise instead to do the research locally, and if you find concrete benefits to an alternative solution, come back, show your work and have a discussion about it then. Merging extra complexity in the form of a plug point without knowing if we're actually going to need it seems risky. On another note, after years of work the stability issues have largely been resolved and L3 HA is in a good state with modern releases of OpenStack. It's not a authoritative solution in the sense that it doesn't cover every possible failure mode, but it covers the major ones and in that sense better than not having any form of HA, and as you pointed out the existing alternatives are not in a better state. The subtext in your email is that now L3 HA is technically where we want it, but some users are resisting adoption because of bad PR or a bad past experience, but not for technical reasons. If that is the case, then perhaps some good PR would be a more cost effective investment than investigating, implementing, stabilizing and maintaining a different backend that will likely take at least a cycle to get merged and another 1 to 2 cycles to iron out kinks. Would you have a critical mass of developers ready to support a pluggable L3 HA now and in the long term? Finally, I can share that L3 HA has been the default in RDO-land for a few cycles now and is being used widely and successfully, in some cases at significant scale. > > [1] - > http://docs.openstack.org/developer/performance-docs/test_results/neutron_features/index.html > [2] - https://bugs.launchpad.net/neutron/+bug/1497272 > https://bugs.launchpad.net/neutron/+bug/1433172 > [3] - https://bugs.launchpad.net/neutron/+bug/1375625 > [4] - http://docs.openstack.org/developer/tooz/ > > > > > -- > Regards, > Ann Taraday > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev