Please, file a bug about that on https://bugs.launchpad.net/neutron/+bugs with tag "l3-ha".
On Thu, Nov 30, 2017 at 6:00 AM Ajay Kalambur (akalambu) <[email protected]> wrote: > I noticed that this happens when the router HA interface shows status: > Down what can cause the ha interface to do down > > Ajay > > > From: Ajay Kalambur <[email protected]> > Reply-To: "OpenStack Development Mailing List (not for usage questions)" < > [email protected]> > Date: Wednesday, November 29, 2017 at 4:14 PM > To: "OpenStack Development Mailing List (not for usage questions)" < > [email protected]> > Subject: [openstack-dev] [neutron] Neutron L3 agent/Keepalived > > Hi > I have a case where after running the system for a while I see that > floating ip association API gets accepted and no Errors in neutron l3 logs > But when I go into the qrouter namespace and check the qg- interface the > floating ip is not added > Also the keepalived.conf is not updated and SIGUP of the keepalived > process is not done > > So whats the place to look in this case. What is the flow in neutron l3 > agent who adds the floating ip to the qg- interface > I see the router update notification received > Key log snippet attached. As you can see SIGUP of keepalived is missing > and also I confirmed keepalived configs are not updated > > > 2017-11-29 14:32:08.883 78 DEBUG oslo_messaging._drivers.amqpdriver [-] > received message with unique_id: b3886b998c3049e799170bb5351300d1 __call__ > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:269 > 2017-11-29 14:32:08.885 78 DEBUG neutron.agent.l3.agent > [req-6c505d32-2d1a-4392-8ea3-0bb888397b9e e759eebecc4648b4964d4ecf439dc0ff > 13b4f6fb188048ba9bbf211344e3342f - - -] Got routers updated notification > :[u'a1a59e5f-d74e-404d-a3aa-1667c4442aed'] routers_updated > /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:413 > 2017-11-29 14:32:08.886 78 DEBUG neutron.agent.l3.agent [-] Starting > router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed, action None, > priority 0 _process_router_update > /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:487 > 2017-11-29 14:32:08.886 78 DEBUG oslo_messaging._drivers.amqpdriver [-] > CALL msg_id: 92eb015c8c84489681b1efbe949fab8b exchange 'neutron' topic > 'q-l3-plugin' _send > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:568 > > 2017-11-29 14:32:09.571 78 DEBUG oslo_messaging._drivers.amqpdriver [-] > received reply msg_id: 92eb015c8c84489681b1efbe949fab8b __call__ > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:416 > 2017-11-29 14:32:09.571 78 DEBUG neutron.callbacks.manager [-] Notify > callbacks [] for router, before_update _notify_loop > /usr/lib/python2.7/site-packages/neutron/callbacks/manager.py:142 > 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.l3.router_info [-] process > router updates process > /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:1105 > 2017-11-29 14:32:09.572 78 DEBUG neutron.agent.linux.utils [-] Running > command (rootwrap daemon): ['ip', 'netns', 'exec', > 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find', '/sys/class/net', > '-maxdepth', '1', '-type', 'l', '-printf', '%f '] execute_rootwrap_daemon > /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105 > 2017-11-29 14:32:09.709 78 DEBUG neutron.agent.linux.utils [-] Exit code: > 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150 > 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock > "l3-agent-pd" acquired by "neutron.agent.linux.pd.sync_router" :: waited > 0.000s inner > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270 > 2017-11-29 14:32:09.710 78 DEBUG oslo_concurrency.lockutils [-] Lock > "l3-agent-pd" released by "neutron.agent.linux.pd.sync_router" :: held > 0.000s inner > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282 > 2017-11-29 14:32:09.710 78 DEBUG neutron.agent.linux.utils [-] Running > command (rootwrap daemon): ['ip', 'netns', 'exec', > 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'find', '/sys/class/net', > '-maxdepth', '1', '-type', 'l', '-printf', '%f '] execute_rootwrap_daemon > /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105 > 2017-11-29 14:32:09.845 78 DEBUG neutron.agent.linux.utils [-] Exit code: > 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150 > 2017-11-29 14:32:09.846 78 DEBUG oslo_concurrency.lockutils [-] Acquired > semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212 > 2017-11-29 14:32:09.846 78 DEBUG neutron.agent.linux.utils [-] Running > command (rootwrap daemon): ['ip', 'netns', 'exec', > 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save'] > execute_rootwrap_daemon > /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105 > 2017-11-29 14:32:09.983 78 DEBUG neutron.agent.linux.utils [-] Exit code: > 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150 > 2017-11-29 14:32:09.986 78 DEBUG neutron.agent.linux.utils [-] Running > command (rootwrap daemon): ['ip', 'netns', 'exec', > 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-restore', '-n'] > execute_rootwrap_daemon > /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105 > 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.utils [-] Exit code: > 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150 > 2017-11-29 14:32:10.120 78 DEBUG neutron.agent.linux.iptables_manager [-] > IPTablesManager.apply completed with success. 7 iptables commands were > issued _apply_synchronized > /usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py:576 > 2017-11-29 14:32:10.121 78 DEBUG oslo_concurrency.lockutils [-] Releasing > semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225 > *2017-11-29 14:32:10.121 78 DEBUG neutron.agent.l3.router_info [-] > Floating ip 5966c8b3-ff5c-4f01-91a6-f81d215bdeca added, status ACTIVE > process_floating_ip_addresses > /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:347* > *2017-11-29 14:32:10.122 78 DEBUG neutron.agent.l3.router_info [-] Sending > floating ip statuses: {u'5966c8b3-ff5c-4f01-91a6-f81d215bdeca': 'ACTIVE'} > update_fip_statuses > /usr/lib/python2.7/site-packages/neutron/agent/l3/router_info.py:915* > 2017-11-29 14:32:10.122 78 DEBUG oslo_messaging._drivers.amqpdriver [-] > CALL msg_id: 18e6ceffec9549a5bd51f65551b65596 exchange 'neutron' topic > 'q-l3-plugin' _send > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:568 > 2017-11-29 14:32:10.150 78 DEBUG oslo_messaging._drivers.amqpdriver [-] > received reply msg_id: 18e6ceffec9549a5bd51f65551b65596 __call__ > /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:416 > 2017-11-29 14:32:10.151 78 DEBUG oslo_concurrency.lockutils [-] Acquired > semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:212 > 2017-11-29 14:32:10.151 78 DEBUG neutron.agent.linux.utils [-] Running > command (rootwrap daemon): ['ip', 'netns', 'exec', > 'qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed', 'iptables-save'] > execute_rootwrap_daemon > /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:105 > 2017-11-29 14:32:10.301 78 DEBUG neutron.agent.linux.utils [-] Exit code: > 0 execute /usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py:150 > 2017-11-29 14:32:10.304 78 DEBUG neutron.agent.linux.iptables_manager [-] > IPTablesManager.apply completed with success. 0 iptables commands were > issued _apply_synchronized > /usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py:576 > 2017-11-29 14:32:10.304 78 DEBUG oslo_concurrency.lockutils [-] Releasing > semaphore "iptables-qrouter-a1a59e5f-d74e-404d-a3aa-1667c4442aed" lock > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225 > 2017-11-29 14:32:10.304 78 DEBUG neutron.callbacks.manager [-] Notify > callbacks [('neutron.agent.linux.pd.update_router--9223372036851355335', > <function update_router at 0x3431398>), > ('neutron.agent.metadata.driver.after_router_updated-3681661', <function > after_router_updated at 0x382d7d0>)] for router, after_update _notify_loop > /usr/lib/python2.7/site-packages/neutron/callbacks/manager.py:142 > 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock > "l3-agent-pd" acquired by "neutron.agent.linux.pd.update_router" :: waited > 0.000s inner > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270 > 2017-11-29 14:32:10.305 78 DEBUG oslo_concurrency.lockutils [-] Lock > "l3-agent-pd" released by "neutron.agent.linux.pd.update_router" :: held > 0.000s inner > /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282 > 2017-11-29 14:32:10.305 78 DEBUG neutron.agent.l3.agent [-] Finished a > router update for a1a59e5f-d74e-404d-a3aa-1667c4442aed > _process_router_update > /usr/lib/python2.7/site-packages/neutron/agent/l3/agent.py:536 > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Regards, Ann Taraday
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
