Public bug reported: It seems that when there are 2 subnets using different segments created on same network, it can cause race condition and one of the green threads will expect tap port to be there but other will treat it as "stale" device and will remove it from namespace.
It causes failures in the neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network pretty often. Stacktrace example: ft1.1: neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network(Open vSwitch Agent)testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true eventlet.sleep(sleep) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 36, in sleep hub.switch() File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() eventlet.timeout.Timeout: 60 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func return f(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 146, in test_multi_segs_network self.vm1 = self._spawn_vm(neutron_port=self.port1) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 65, in _spawn_vm vm.block_until_dhcp_config_done() File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/machine.py", line 217, in block_until_dhcp_config_done utils.wait_until_true( File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 747, in wait_until_true raise exception neutron.tests.common.machine_fixtures.FakeMachineException: Address 10.0.11.195/24 or gateway 10.0.11.1 not configured properly on port portbc2128 Failure example: https://750440148c983d142b4c-dd0601cbd7f0aa15c09a82437ec7e47b.ssl.cf5.rackcdn.com/872396/2/gate/neutron- fullstack-with-uwsgi/c5e45ed/testr_results.html ** Affects: neutron Importance: Critical Assignee: Slawek Kaplonski (slaweq) Status: Confirmed ** Tags: gate-failure l3-ipam-dhcp -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/2007152 Title: Race condition in DHCP agent cause deletion of the dhcp port as stale port Status in neutron: Confirmed Bug description: It seems that when there are 2 subnets using different segments created on same network, it can cause race condition and one of the green threads will expect tap port to be there but other will treat it as "stale" device and will remove it from namespace. It causes failures in the neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network pretty often. Stacktrace example: ft1.1: neutron.tests.fullstack.test_multisegs.TestMultiSegs.test_multi_segs_network(Open vSwitch Agent)testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 743, in wait_until_true eventlet.sleep(sleep) File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/greenthread.py", line 36, in sleep hub.switch() File "/home/zuul/src/opendev.org/openstack/neutron/.tox/dsvm-fullstack-gate/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 313, in switch return self.greenlet.switch() eventlet.timeout.Timeout: 60 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/base.py", line 182, in func return f(self, *args, **kwargs) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 146, in test_multi_segs_network self.vm1 = self._spawn_vm(neutron_port=self.port1) File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/test_multisegs.py", line 65, in _spawn_vm vm.block_until_dhcp_config_done() File "/home/zuul/src/opendev.org/openstack/neutron/neutron/tests/fullstack/resources/machine.py", line 217, in block_until_dhcp_config_done utils.wait_until_true( File "/home/zuul/src/opendev.org/openstack/neutron/neutron/common/utils.py", line 747, in wait_until_true raise exception neutron.tests.common.machine_fixtures.FakeMachineException: Address 10.0.11.195/24 or gateway 10.0.11.1 not configured properly on port portbc2128 Failure example: https://750440148c983d142b4c-dd0601cbd7f0aa15c09a82437ec7e47b.ssl.cf5.rackcdn.com/872396/2/gate/neutron- fullstack-with-uwsgi/c5e45ed/testr_results.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/2007152/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp