Public bug reported: OpenStack Queens from UCA (xenial, GA kernel), 2 external subnets (one routed provider network), 2 tenant subnets, all subnets in the same address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24 Other tenant subnet cidr: 192.168.200.0/24 Relevant agent configs: http://paste.openstack.org/show/718514/ Commands and outputs: http://paste.openstack.org/show/JFYmGJwF1pdtliQOfXgd/ Overall, a similar situation as with https://bugs.launchpad.net/neutron/+bug/1759956 but with one tenant subnet at first for which routes and rules do not get deleted at all. Problem description: * router add subnet tenantsubnet * routes in fip namespace and rules in qrouter namespace get created and a distributed port gets created for DVR; * router remove subnet tenantsubnet * routes are still there, no new logged events in DVR l3 agent logs If two networks are added then removing one of them triggers removal of routes and rules and new messages are logged in l3 agent log (the rules removed are affected by pad.lv/1759956). A sequence of add subnet/remove subnet commands may result in errors logged in l3 agent logs: http://paste.openstack.org/show/718511/ Sometimes after re-adding a tenantsubnet in presence of othertenantsubnet a proper route is added for a few seconds but then removed: # just do some operations (openstack) router add subnet pubrouter tenantsubnet (openstack) router add subnet pubrouter othertenantsubnet (openstack) router add subnet pubrouter tenantsubnet (openstack) router add subnet pubrouter tenantsubnet (openstack) router remove subnet pubrouter tenantsubnet # lots of errors, see http://paste.openstack.org/show/718511/ # try again without restarting agents (openstack) router add subnet pubrouter tenantsubnet # ran client command # ... got 192.168.100.0/24 here for a few seconds while l3 agent was doing something 10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5 169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115 192.168.100.0/24 via 169.254.106.114 dev fpr-3182a7c6-b 192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b # finished server and l3 agent finished processing "router add subnet pubrouter tenantsubnet" # route got deleted root@ipotane:~# ip netns exec fip-64ab1ec3-4927-4f09-87f9-804e7f4f8748 ip r 10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5 169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115 192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b There is something wrong with how tenant network add/remove notifications are sent it seems because on first removal of a tenant network nothing is logged in l3 agent logs but there is activity in neutron server logs. ** Affects: neutron Importance: Undecided Status: New ** Tags: cpe-onsite -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1761556 Title: [dvr][fast-exit] router add/remove subnet operations are not idempotent Status in neutron: New Bug description: OpenStack Queens from UCA (xenial, GA kernel), 2 external subnets (one routed provider network), 2 tenant subnets, all subnets in the same address scope to trigger "fast exit" logic. Tenant subnet cidr: 192.168.100.0/24 Other tenant subnet cidr: 192.168.200.0/24 Relevant agent configs: http://paste.openstack.org/show/718514/ Commands and outputs: http://paste.openstack.org/show/JFYmGJwF1pdtliQOfXgd/ Overall, a similar situation as with https://bugs.launchpad.net/neutron/+bug/1759956 but with one tenant subnet at first for which routes and rules do not get deleted at all. Problem description: * router add subnet tenantsubnet * routes in fip namespace and rules in qrouter namespace get created and a distributed port gets created for DVR; * router remove subnet tenantsubnet * routes are still there, no new logged events in DVR l3 agent logs If two networks are added then removing one of them triggers removal of routes and rules and new messages are logged in l3 agent log (the rules removed are affected by pad.lv/1759956). A sequence of add subnet/remove subnet commands may result in errors logged in l3 agent logs: http://paste.openstack.org/show/718511/ Sometimes after re-adding a tenantsubnet in presence of othertenantsubnet a proper route is added for a few seconds but then removed: # just do some operations (openstack) router add subnet pubrouter tenantsubnet (openstack) router add subnet pubrouter othertenantsubnet (openstack) router add subnet pubrouter tenantsubnet (openstack) router add subnet pubrouter tenantsubnet (openstack) router remove subnet pubrouter tenantsubnet # lots of errors, see http://paste.openstack.org/show/718511/ # try again without restarting agents (openstack) router add subnet pubrouter tenantsubnet # ran client command # ... got 192.168.100.0/24 here for a few seconds while l3 agent was doing something 10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5 169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115 192.168.100.0/24 via 169.254.106.114 dev fpr-3182a7c6-b 192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b # finished server and l3 agent finished processing "router add subnet pubrouter tenantsubnet" # route got deleted root@ipotane:~# ip netns exec fip-64ab1ec3-4927-4f09-87f9-804e7f4f8748 ip r 10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5 169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115 192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b There is something wrong with how tenant network add/remove notifications are sent it seems because on first removal of a tenant network nothing is logged in l3 agent logs but there is activity in neutron server logs. To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1761556/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
