** Also affects: neutron/juno
Importance: Undecided
Status: New
** Changed in: neutron/juno
Status: New => Fix Committed
** Changed in: neutron/juno
Milestone: None => 2014.2.2
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1404945
Title:
Default gateway can vanish from HA routers, destroying external
connectivity for all VMs on that network
Status in OpenStack Neutron (virtual network service):
Fix Committed
Status in neutron juno series:
Fix Committed
Bug description:
The default gateway can vanish from the HA router namespace after
certain operations.
My setup:
Fedora 20
keepalived-1.2.13-1.fc20.x86_64
Network manager turned off.
I can reproduce this reliably on my system, but cannot reproduce this
on a RHEL 7 system. Even on that system, the issue manifests on its
own, I just can't reproduce it at will.
How I reproduce on my system:
Create an HA router
Set it as a gateway
Go to the master instance
Observe that the namespace has a default gateway
Add an internal interface (Make sure that the IP is 'lower' than the IP of
the external interface, this is explained below)
Default gateway will no longer exist
Cause:
keepalived.conf has two sections for VIPs: virtual_ipaddress, and
virtual_ipaddress_excluded. The difference is that any VIPs that go in the
first section will be propagated on the wire, and any VIPs in the excluded
section do not. Traditional configuration of keepalived places one VIP in the
normal section, henceforth known as the 'primary VIP', and all other VIPs in
the excluded section. Currently the keepalived manager does this by sorting the
VIPs (Internal IPs, external SNAT IP, and all floating IPs), placing the lowest
one (By string comparison) as the primary, and the rest of the VIPs in the
excluded section:
https://github.com/openstack/neutron/blob/master/neutron/agent/linux/keepalived.py#L155
That code is ran, and keepalived.conf is built when ever a router is
updated. This means that the primary VIP can change on router updates.
As it turns out, after a conversation with a keepalived developer,
keepalived assumes that the order does not change (This is possibly a
keepalived bug, depending on your view on life, the ordering of the
stars when keepalived is executed and the wind speed in the Falkland
Islands in the past leap year). On my system, with the currently
installed keepalived version, whenever the primary VIP changes, the
default gateway (Present in the virtual_routes section of
keepalived.conf) is violently removed.
Possible solution:
Make sure that the primary VIP never changes. For example: Fabricate an IP
per HA router cluster (Derived from the VRID?), add it as a VIP on the HA
device, configure it as the primary VIP. I played around with a hacky variation
of this solution and I could no longer reproduce the issue.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1404945/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
