[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
** Changed in: tacker Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in OpenStack Compute (nova): Fix Released Status in tacker: Fix Released Status in nova package in Ubuntu: Fix Released Status in nova source package in Xenial: Fix Released Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
Also marking this as fix-released for Ubuntu since it is available in nova 13.0.0+. ** Changed in: nova (Ubuntu) Status: Confirmed => Fix Released ** Also affects: nova (Ubuntu Xenial) Importance: Undecided Status: New ** Changed in: nova (Ubuntu Xenial) Status: New => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in OpenStack Compute (nova): Fix Released Status in tacker: Fix Committed Status in nova package in Ubuntu: Fix Released Status in nova source package in Xenial: Fix Released Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
** No longer affects: heat -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in OpenStack Compute (nova): Fix Released Status in tacker: Fix Committed Status in nova package in Ubuntu: New Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
** Also affects: heat (Ubuntu) Importance: Undecided Status: New ** Also affects: nova (Ubuntu) Importance: Undecided Status: New ** No longer affects: heat (Ubuntu) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in heat: New Status in OpenStack Compute (nova): Fix Released Status in tacker: Fix Committed Status in nova package in Ubuntu: New Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/heat/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
Acording to the abandoned review [1] (from the last Nova assignee yong sheng gong) this got fixed with patch [2]. Patch [2] didn't mention this in its commit message, that's why this bug report looks like it is still open. I'm closing it manually. References: [1] https://review.openstack.org/#/c/252824/ [2] https://review.openstack.org/#/c/252565/ ** Changed in: nova Status: In Progress => Fix Released ** Tags added: liberty-backport-potential -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in heat: New Status in OpenStack Compute (nova): Fix Released Status in tacker: Fix Committed Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/heat/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1515768] Re: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy
** Attachment added: "h-eng.log.2015-11-25-193438" https://bugs.launchpad.net/tacker/+bug/1515768/+attachment/4525683/+files/h-eng.log.2015-11-25-193438 ** Also affects: heat Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1515768 Title: Instance creation fails with libvirtError: Unable to create tap device: Device or resource busy Status in heat: New Status in neutron: New Status in OpenStack Compute (nova): New Status in tacker: New Bug description: Summary: This issue is observed frequently on Jenkins gate and has been reproducible in local setup too. Steps: Initiate 3 stack create requests at once in a script: heat stack-create -f /home/stack/template_file stack1 heat stack-create -f /home/stack/template_file stack2 heat stack-create -f /home/stack/template_file stack3 using the following HOT file: http://paste.openstack.org/show/479920/ One of the stack creations fails with CreateFailed: Resource Create Failed: Conflict: Resources. vdu3: Port Is Still In Use. From the nova logs, there are duplicate bridges created for one of the servers. The qemu xml fails with libvirtError: Unable to create tap device tapd3a3d9e9-5d: Device or resource busy. See timestamp 2015-11-25 23:03:14.940 in n-cpu.log Attaching the relevant n-cpu.log, q-svc.log and h-eng.log Observation: The 1st network interface for the nova instance is a Neutron Port resource provided in HOT template. Nova sends a PUT request to update the port information. It also sends 2 POST requests for the 2nd and 3rd network interfaces. Neutron receives the PUT request and sends network-event changed event while nova is still waiting for the POST response for the 2 ports. If the network-changed event is received before the 3rd port POST response is received, refresh_cache is acquired by nova_service Nova sends a query for port information, updates the cache and release the lock. By then, POST requests are completed which acquires the cache lock again and sends request for network info. refresh_cache is updated twice and contains duplicate set of ports Network vifs are built for all 6 ports and qemu xml is build based on that. Duplicate bridges in xml is complained by libvirt as device or resource busy. Version and environment: Devstack Master To manage notifications about this bug go to: https://bugs.launchpad.net/heat/+bug/1515768/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
