Public bug reported: After parsing a lot of log files related to check failure, looks like the q-vpn at the time when I would expect to add floating ip , it destroys the router's qg- and qr- interfaces.
However after the floating ip deletion request the q-vpn service restores the qg-, qr- interfaces. tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,image,network,volume] failed in http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/console.html. admin user: admin/9e02f14321454af6bb27587770f27d9b admin tenant id: admin/413bb1232bca45069f3a3256839effa1 test user: TestMinimumBasicScenario-1306090821/c8dd95056c0b407e8dd168dbf410a66a test Tenant: TestMinimumBasicScenario-993819377/2527b8222e3343bca9f70343e608880c External Net : public/c29040d3-7e73-4a87-9f73-bb5cbe602afb External subnet: public-subnet/ee754eb6-6194-4a25-a4cc-f9233d366c1e Network: TestMinimumBasicScenario-1375749858-network/f029d7a8-54e0-484c-a215-cc34066ae830 Subnet: TestMinimumBasicScenario-1375749858-subnet/7edc72be-1207-4571-95d4-911223885ae7 10.100.0.0/28 Router id:TestMinimumBasicScenario-1375749858-router/08216822-5ee2-4313-be7e-dad2d84147db Expected interfaces in the qrouter-08216822-5ee2-4313-be7e-dad2d84147db: * lo 127.0.0.1 * qr-529eddd4-2c 10.100.0.1/28 iface_id: 529eddd4-2ca8-43ec-9cab-29c3a6632604, attached-mac: fa:16:3e:2a:f8:ba, ofport 166 * qg-9be8f502-93 172.24.4.85/24 iface-id: 9be8f502-9360-47dc-9eff-33c8743e7c2b attached-mac: fa:16:3e:be:a1:54, ofport 37 Floating IP: 172.24.4.87 (Never appears in the q-vpn log) port: (net/subnet/port)(c29040d3-7e73-4a87-9f73-bb5cbe602afb/ee754eb6-6194-4a25-a4cc-f9233d366c1e/013b0b2d-80ed-403d-b380-6b6895ce34f5) mac?: fa:16:3e:15:f2:57 floating ip uuid: cd84111e-af6a-4c26-af73-3167419c664a Instance: Ipv4: 10.100.0.2 mac: FA:16:3E:18:F1:69 Instance uuid: 8a552eda-2fbd-4972-bfcf-cee7e6472871 iface_id/port_id: 4188c532-3265-4294-8b4e-9bbfe5a482e8 ovsdb_interface_uuid: 9d7b858b-745e-482b-b91a-1e9ae34fc545 intbr tag: 49 dhcp server dev: tap4c6c6e06-e4 ns: qdhcp-f029d7a8-54e0-484c-a215-cc34066ae830 ip: 10.100.0.3 mac: fa:16:3e:a2:f1:ea intbr tag: 49 Host: eth0: 10.7.16.229/15 mac: 02:16:3e:52:5d:ff Router + router interface creation in the logs: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_49_46_924 http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-svc.txt.gz#_2014-05-01_19_49_46_724 Floating IP create: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_50_18_447 Floating IP associate: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_50_18_814 q-svc starts destroying the router: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-vpn.txt.gz#_2014-05-01_19_50_20_277 Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-c0cd93e7-5cb0-403b-a509-8e07b352b89d', 'ipsec', 'whack', '--ctlbase', '/opt/stack/data/neutron/ipsec/c0cd93e7-5cb0-403b-a509-8e07b352b89d/var/run/pluto', '--status'] Exit code: 1 Stdout: '' Stderr: 'whack: Pluto is not running (no "/opt/stack/data/neutron/ipsec/c0cd93e7-5cb0-403b-a509-8e07b352b89d/var/run/pluto.ctl")\n' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.277 19279 DEBUG neutron.openstack.common.lockutils [-] Semaphore / lock released "sync" inner /opt/stack/new/neutron/neutron/openstack/common/lockutils.py:252 2014-05-01 19:50:20.277 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-08216822-5ee2-4313-be7e-dad2d84147db', 'ip', '-o', 'link', 'show', 'qr-529eddd4-2c'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:20.516 19279 DEBUG neutron.agent.linux.utils [-] Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-08216822-5ee2-4313-be7e-dad2d84147db', 'ip', '-o', 'link', 'show', 'qr-529eddd4-2c'] Exit code: 0 Stdout: '605: qr-529eddd4-2c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\ link/ether fa:16:3e:2a:f8:ba brd ff:ff:ff:ff:ff:ff\n' Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.517 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'br-int'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:20.533 19279 DEBUG neutron.agent.linux.utils [-] Command: ['ip', '-o', 'link', 'show', 'br-int'] Exit code: 0 Stdout: '6: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN \\ link/ether c6:49:9f:72:d2:4a brd ff:ff:ff:ff:ff:ff\n' Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.534 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-port', 'br-int', 'qr-529eddd4-2c'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:21.061 19279 DEBUG neutron.agent.linux.utils [-] Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-port', 'br-int', 'qr-529eddd4-2c'] Exit code: 0 Tempest tries to connect to the VM: 2014-05-01 19:50:19,463 .. 2014-05-01 19:53:48,494 ~ 209 sec Delete request for the floatingip: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_54_24_191 http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_54_25_047 Neutron touches the qr-529eddd4-2c interface + port, http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-vpn.txt.gz#_2014-05-01_19_54_37_886 Delete request for router and router interface: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_54_57_423 ** Affects: neutron Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1315467 Title: Neutron deletes the router interface instead of adding a floatingip Status in OpenStack Neutron (virtual network service): New Bug description: After parsing a lot of log files related to check failure, looks like the q-vpn at the time when I would expect to add floating ip , it destroys the router's qg- and qr- interfaces. However after the floating ip deletion request the q-vpn service restores the qg-, qr- interfaces. tempest.scenario.test_minimum_basic.TestMinimumBasicScenario.test_minimum_basic_scenario[compute,image,network,volume] failed in http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/console.html. admin user: admin/9e02f14321454af6bb27587770f27d9b admin tenant id: admin/413bb1232bca45069f3a3256839effa1 test user: TestMinimumBasicScenario-1306090821/c8dd95056c0b407e8dd168dbf410a66a test Tenant: TestMinimumBasicScenario-993819377/2527b8222e3343bca9f70343e608880c External Net : public/c29040d3-7e73-4a87-9f73-bb5cbe602afb External subnet: public-subnet/ee754eb6-6194-4a25-a4cc-f9233d366c1e Network: TestMinimumBasicScenario-1375749858-network/f029d7a8-54e0-484c-a215-cc34066ae830 Subnet: TestMinimumBasicScenario-1375749858-subnet/7edc72be-1207-4571-95d4-911223885ae7 10.100.0.0/28 Router id:TestMinimumBasicScenario-1375749858-router/08216822-5ee2-4313-be7e-dad2d84147db Expected interfaces in the qrouter-08216822-5ee2-4313-be7e-dad2d84147db: * lo 127.0.0.1 * qr-529eddd4-2c 10.100.0.1/28 iface_id: 529eddd4-2ca8-43ec-9cab-29c3a6632604, attached-mac: fa:16:3e:2a:f8:ba, ofport 166 * qg-9be8f502-93 172.24.4.85/24 iface-id: 9be8f502-9360-47dc-9eff-33c8743e7c2b attached-mac: fa:16:3e:be:a1:54, ofport 37 Floating IP: 172.24.4.87 (Never appears in the q-vpn log) port: (net/subnet/port)(c29040d3-7e73-4a87-9f73-bb5cbe602afb/ee754eb6-6194-4a25-a4cc-f9233d366c1e/013b0b2d-80ed-403d-b380-6b6895ce34f5) mac?: fa:16:3e:15:f2:57 floating ip uuid: cd84111e-af6a-4c26-af73-3167419c664a Instance: Ipv4: 10.100.0.2 mac: FA:16:3E:18:F1:69 Instance uuid: 8a552eda-2fbd-4972-bfcf-cee7e6472871 iface_id/port_id: 4188c532-3265-4294-8b4e-9bbfe5a482e8 ovsdb_interface_uuid: 9d7b858b-745e-482b-b91a-1e9ae34fc545 intbr tag: 49 dhcp server dev: tap4c6c6e06-e4 ns: qdhcp-f029d7a8-54e0-484c-a215-cc34066ae830 ip: 10.100.0.3 mac: fa:16:3e:a2:f1:ea intbr tag: 49 Host: eth0: 10.7.16.229/15 mac: 02:16:3e:52:5d:ff Router + router interface creation in the logs: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_49_46_924 http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-svc.txt.gz#_2014-05-01_19_49_46_724 Floating IP create: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_50_18_447 Floating IP associate: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_50_18_814 q-svc starts destroying the router: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-vpn.txt.gz#_2014-05-01_19_50_20_277 Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-c0cd93e7-5cb0-403b-a509-8e07b352b89d', 'ipsec', 'whack', '--ctlbase', '/opt/stack/data/neutron/ipsec/c0cd93e7-5cb0-403b-a509-8e07b352b89d/var/run/pluto', '--status'] Exit code: 1 Stdout: '' Stderr: 'whack: Pluto is not running (no "/opt/stack/data/neutron/ipsec/c0cd93e7-5cb0-403b-a509-8e07b352b89d/var/run/pluto.ctl")\n' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.277 19279 DEBUG neutron.openstack.common.lockutils [-] Semaphore / lock released "sync" inner /opt/stack/new/neutron/neutron/openstack/common/lockutils.py:252 2014-05-01 19:50:20.277 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-08216822-5ee2-4313-be7e-dad2d84147db', 'ip', '-o', 'link', 'show', 'qr-529eddd4-2c'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:20.516 19279 DEBUG neutron.agent.linux.utils [-] Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qrouter-08216822-5ee2-4313-be7e-dad2d84147db', 'ip', '-o', 'link', 'show', 'qr-529eddd4-2c'] Exit code: 0 Stdout: '605: qr-529eddd4-2c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN \\ link/ether fa:16:3e:2a:f8:ba brd ff:ff:ff:ff:ff:ff\n' Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.517 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['ip', '-o', 'link', 'show', 'br-int'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:20.533 19279 DEBUG neutron.agent.linux.utils [-] Command: ['ip', '-o', 'link', 'show', 'br-int'] Exit code: 0 Stdout: '6: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN \\ link/ether c6:49:9f:72:d2:4a brd ff:ff:ff:ff:ff:ff\n' Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:74 2014-05-01 19:50:20.534 19279 DEBUG neutron.agent.linux.utils [-] Running command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-port', 'br-int', 'qr-529eddd4-2c'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:48 2014-05-01 19:50:21.061 19279 DEBUG neutron.agent.linux.utils [-] Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=10', '--', '--if-exists', 'del-port', 'br-int', 'qr-529eddd4-2c'] Exit code: 0 Tempest tries to connect to the VM: 2014-05-01 19:50:19,463 .. 2014-05-01 19:53:48,494 ~ 209 sec Delete request for the floatingip: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_54_24_191 http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-n-api.txt.gz#_2014-05-01_19_54_25_047 Neutron touches the qr-529eddd4-2c interface + port, http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/screen-q-vpn.txt.gz#_2014-05-01_19_54_37_886 Delete request for router and router interface: http://logs.openstack.org/79/88579/2/check/check-tempest-dsvm-neutron-full/f76ee0e/logs/tempest.txt.gz#_2014-05-01_19_54_57_423 To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1315467/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
