While I agree that it would be way more user friendly to give a warning/error in the problematic API workflow that would entail some cross project changes because today:
* nova does not know when an already bound port is added to a trunk * neutron does not know if nova is supposed to auto-delete a port That means neither nova nor neutron can detect the error condition in itself. Again, I believe changing the workflow to pre-create the parent port for the server stops the problem described in this bug report completely. So I'm setting this bug as Invalid. But let me know if you see other alternatives. ** Changed in: neutron Status: New => Invalid -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1878031 Title: Unable to delete an instance | Conflict: Port [port-id] is currently a parent port for trunk [trunk-id] Status in neutron: Invalid Bug description: When you create a trunk in Neutron you create a parent port for the trunk and attach the trunk to the parent. Then subports can be created on the trunk. When instances are created on the trunk, first a port is created and then an instance is associated with a free port. It looks to me that's this is the oversight in the logic. From the perspective of the code, the parent port looks like any other port attached to the trunk bridge. It doesn't have an instance attached to it so it looks like it's not being used for anything (which is technically correct). So it becomes an eligible port for an instance to bind to. That is all fine and dandy until you go to delete the instance and you get the "Port [port-id] is currently a parent port for trunk [trunk-id]" exception just as happened here. Anecdotally, it's seems rare that an instance will actually bind to it, but that is what happened for the user in this case and I have had several pings over the past year about people in a similar state. I propose that when a port is made parent port for a trunk, that the trunk be established as the owner of the port. That way it will be ineligible for instances seeking to bind to the port. See also old bug: https://bugs.launchpad.net/neutron/+bug/1700428 Description of problem: Attempting to delete instance failed with error in nova-compute ~~~ 2020-03-04 09:52:46.257 1 WARNING nova.network.neutronv2.api [req-0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Failed to delete port 991e4e50-481a-4ca6-9ea6-69f848c4ca9f for instance.: Conflict: Port 991e4e50-481a-4ca6-9ea6-69f848c4ca9f is currently a parent port for trunk 5800ee0f-b558-46cb-bb0b-92799dbe02cf. ~~~ ~~~ [stack@migration-host ~]$ openstack network trunk show 5800ee0f-b558-46cb-bb0b-92799dbe02cf +-----------------+--------------------------------------+ | Field | Value | +-----------------+--------------------------------------+ | admin_state_up | UP | | created_at | 2020-03-04T09:01:23Z | | description | | | id | 5800ee0f-b558-46cb-bb0b-92799dbe02cf | | name | WIN-TRUNK | | port_id | 991e4e50-481a-4ca6-9ea6-69f848c4ca9f | | project_id | 695c417810ac460480055b074bc41817 | | revision_number | 3 | | status | ACTIVE | | sub_ports | | | tags | [] | | tenant_id | 695c417810ac460480055b074bc41817 | | updated_at | 2020-03-04T10:20:46Z | +-----------------+--------------------------------------+ [stack@migration-host ~]$ nova interface-list 2f9e3740-b425-4f00-a949-e1aacf2239c4 +------------+--------------------------------------+--------------------------------------+--------------+-------------------+ | Port State | Port ID | Net ID | IP addresses | MAC Addr | +------------+--------------------------------------+--------------------------------------+--------------+-------------------+ | DOWN | 991e4e50-481a-4ca6-9ea6-69f848c4ca9f | 9be62c82-4274-48b4-bba0-39ccbdd5bb1b | 192.168.0.19 | fa:16:3e:0a:2b:9b | +------------+--------------------------------------+--------------------------------------+--------------+-------------------+ [stack@migration-host ~]$ openstack port show 991e4e50-481a-4ca6-9ea6-69f848c4ca9f +-----------------------+---------------------------------------------------------------------------------------------------+ | Field | Value | +-----------------------+---------------------------------------------------------------------------------------------------+ | admin_state_up | UP | | allowed_address_pairs | | | binding_host_id | cnibydc01cmp1.pl.cni.local | | binding_profile | | | binding_vif_details | port_filter='True' | | binding_vif_type | ovs | | binding_vnic_type | normal | | created_at | 2020-03-03T12:21:34Z | | data_plane_status | None | | description | | | device_id | 2f9e3740-b425-4f00-a949-e1aacf2239c4 | | device_owner | compute:nested | | dns_assignment | fqdn='host-192-168-0-19.openstacklocal.', hostname='host-192-168-0-19', ip_address='192.168.0.19' | | dns_name | | | extra_dhcp_opts | | | fixed_ips | ip_address='192.168.0.19', subnet_id='efe88dec-368f-4e0e-b94a-5889915704a6' | | id | 991e4e50-481a-4ca6-9ea6-69f848c4ca9f | | ip_address | None | | mac_address | fa:16:3e:0a:2b:9b | | name | | | network_id | 9be62c82-4274-48b4-bba0-39ccbdd5bb1b | | option_name | None | | option_value | None | | port_security_enabled | True | | project_id | 695c417810ac460480055b074bc41817 | | qos_policy_id | None | | revision_number | 26 | | security_group_ids | | | status | DOWN | | subnet_id | None | | tags | | | trunk_details | {u'trunk_id': u'5800ee0f-b558-46cb-bb0b-92799dbe02cf', u'sub_ports': []} | | updated_at | 2020-03-05T07:03:12Z | +-----------------------+----------------------------------------------------- ~~~ Detailed breakdown of an instance where this occurred. 1. I do see 2 interfaces were attached during instance deletion. i. First interface which we still see as attached,port-id=991e4e50-481a-4ca6-9ea6-69f848c4ca9f. ~~~ 2020-03-04 09:52:44.425 1 INFO os_vif [req-0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:0a:2b:9b,bridge_name='br-int',has_traffic_filtering=True,id=991e4e50-481a-4ca6-9ea6- 69f848c4ca9f,network=Network(9be62c82-4274-48b4-bba0- 39ccbdd5bb1b),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap991e4e50-48'). ii. 2nd interface which seems deleted with port-id=64f6a222-9e9a-4494-bf3f-bc0ef72b9ec2 & network-id=90246055-5fce-4526-bdb0- 5e6466dd7946. ~~~ 2020-03-04 09:52:44.427 1 INFO os_vif [req-0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] Successfully unplugged vif VIFOpenVSwitch(active=True,address=fa:16:3e:47:41:c8,bridge_name='br-int',has_traffic_filtering=True,id=64f6a222-9e9a-4494- bf3f-bc0ef72b9ec2,network=Network(90246055-5fce-4526-bdb0-5e6466dd7946),plugin='ovs' ,port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tap64f6a222-9e') ~~~ 2. Neutron tried to detach & delete interface-64f6a222-9e9a-4494 -bf3f-bc0ef72b9ec2. ~~~ 2020-03-04 09:52:44.580 1 INFO nova.compute.manager [req-0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Took 1.45 seconds to destroy the instance on the hypervisor. 2020-03-04 09:52:46.086 1 INFO nova.compute.manager [req-ac6ede7a-5b8f-45af-8239-d3003ad56f50 f378f367ea774fc7bf7e17c160141f85 0a0aed879b1149e18c8e80f5b1379984 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Neutron deleted interface 64f6a222-9e9a-4494-bf3f-bc0ef72b9ec2; detaching it from the instance and deleting it from the info cache. ~~~~ 3. But at the next moment interface detach was failed since the instance was already removed from the hypervisor. ~~~ 2020-03-04 09:52:46.106 1 WARNING nova.compute.manager [req-ac6ede7a-5b8f-45af-8239-d3003ad56f50 f378f367ea774fc7bf7e17c160141f85 0a0aed879b1149e18c8e80f5b1379984 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Detach interface failed, port_id=64f6a222-9e9a-4494-bf3f-bc0ef72b9ec2, reason: Instance 2f9e3740-b425-4f00-a949-e1aacf2239c4 could not be found.: InstanceNotFound: Instance 2f9e3740-b425-4f00-a949-e1aacf2239c4 could not be found. ~~~ 4. At the next moment, we have below error ~~~~ 2020-03-04 09:52:46.257 1 WARNING nova.network.neutronv2.api [req-0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Failed to delete port 991e4e50-481a-4ca6-9ea6-69f848c4ca9f for instance.: Conflict: Port 991e4e50-481a-4ca6-9ea6-69f848c4ca9f is currently a parent port for trunk 5800ee0f-b558-46cb-bb0b-92799dbe02cf. ~~~ [1] ~~~ 2020-03-04 09:52:46.257 1 WARNING nova.network.neutronv2.api [req- 0dd45fe4-861c-46d3-a5ec-7db36352da58 02c6d1bc10fe4ffaa289c786cd09b146 695c417810ac460480055b074bc41817 - default default] [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Failed to delete port 991e4e50 -481a-4ca6-9ea6-69f848c4ca9f for instance.: Conflict: Port 991e4e50 -481a-4ca6-9ea6-69f848c4ca9f is currently a parent port for trunk 5800ee0f-b558-46cb-bb0b-92799dbe02cf. Neutron server returns request_ids: ['req-dbd7a924-a9d2-4da5-aa41-b930580ad4f2'] 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Traceback (most recent call last): 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 1237, in _delete_ports 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] neutron.delete_port(port) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 115, in wrapper 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] ret = obj(*args, **kwargs) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 803, in delete_port 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] return self.delete(self.port_path % (port)) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 115,in wrapper 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] ret = obj(*args, **kwargs) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 350, in delete 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] headers=headers, params=params) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 115,in wrapper 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] ret = obj(*args, **kwargs) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 331, in retry_request 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] headers=headers, params=params) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 115,in wrapper 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] ret = obj(*args, **kwargs) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 294, in do_request 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] self._handle_fault_response(status_code, replybody, resp) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/nova/network/neutronv2/api.py", line 115,in wrapper 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] ret = obj(*args, **kwargs) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 269, in _handle_fault_response 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] exception_handler_v20(status_code, error_body) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] File "/usr/lib/python2.7/site-packages/neutronclient/v2_0/client.py", line 93, in exception_handler_v20 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] request_ids=request_ids) 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Conflict: Port 991e4e50-481a-4ca6-9ea6-69f848c4ca9f is currently a parent port for trunk 5800ee0f-b558-46cb-bb0b-92799dbe02cf. 2020-03-04 09:52:46.257 1 ERROR nova.network.neutronv2.api [instance: 2f9e3740-b425-4f00-a949-e1aacf2239c4] Neutron server returns request_ids: ['req-dbd7a924-a9d2-4da5-aa41-b930580ad4f2'] ~~~ To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1878031/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp