** Also affects: nova/queens Importance: Undecided Status: New ** Changed in: nova Status: New => Invalid
** Changed in: nova/queens Status: New => In Progress ** Changed in: nova/queens Importance: Undecided => Medium ** Changed in: nova/queens Assignee: (unassigned) => Stephen Finucane (stephenfinucane) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1821733 Title: Failed to compute_task_build_instances: local variable 'sibling_set' referenced before assignment Status in OpenStack Compute (nova): Invalid Status in OpenStack Compute (nova) queens series: In Progress Bug description: Reproduced from rhbz#1686511 (https://bugzilla.redhat.com/show_bug.cgi?id=1686511) When spawning an Openstack instance, this error is received: 2019-03-07 08:07:38.499 3124 WARNING nova.scheduler.utils [req-e577cf31-7a58-420f-8ba5-3f962569ab08 0c90c8d8b42c42e883d2135cc733cac4 8b869a98a43e4fc48001e0ff6d149fe6 - - -] Failed to compute_task_build_instances: local variable 'sibling_set' referenced before assignment Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 133, in _process_incoming res = self.dispatcher.dispatch(message) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 150, in dispatch return self._do_dispatch(endpoint, method, ctxt, args) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 121, in _do_dispatch result = func(ctxt, **new_args) File "/usr/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 199, in inner return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/nova/scheduler/manager.py", line 104, in select_destinations dests = self.driver.select_destinations(ctxt, spec_obj) File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 53, in select_destinations selected_hosts = self._schedule(context, spec_obj) File "/usr/lib/python2.7/site-packages/nova/scheduler/filter_scheduler.py", line 113, in _schedule spec_obj, index=num) File "/usr/lib/python2.7/site-packages/nova/scheduler/host_manager.py", line 576, in get_filtered_hosts hosts, spec_obj, index) File "/usr/lib/python2.7/site-packages/nova/filters.py", line 89, in get_filtered_objects list_objs = list(objs) File "/usr/lib/python2.7/site-packages/nova/filters.py", line 44, in filter_all if self._filter_one(obj, spec_obj): File "/usr/lib/python2.7/site-packages/nova/scheduler/filters/__init__.py", line 44, in _filter_one return self.host_passes(obj, spec) File "/usr/lib/python2.7/site-packages/nova/scheduler/filters/numa_topology_filter.py", line 123, in host_passes pci_stats=host_state.pci_stats)) File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 1297, in numa_fit_instance_to_host host_cell, instance_cell, limits) File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 906, in _numa_fit_instance_cell host_cell, instance_cell) File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 854, in _numa_fit_instance_cell_with_pinning max(map(len, host_cell.siblings))) File "/usr/lib/python2.7/site-packages/nova/virt/hardware.py", line 805, in _pack_instance_onto_cores itertools.chain(*sibling_set))) UnboundLocalError: local variable 'sibling_set' referenced before assignment 2019-03-07 08:07:38.500 3124 WARNING nova.scheduler.utils [req- e577cf31-7a58-420f-8ba5-3f962569ab08 0c90c8d8b42c42e883d2135cc733cac4 8b869a98a43e4fc48001e0ff6d149fe6 - - -] [instance: 5bca186a-5a36-4b0f- 8b7a-f2f3bc168b29] Setting instance to ERROR state. This issues appears to be because of: https://github.com/openstack/nova/blob/da9f9c962fe00dbfc9c8fe9c47e964816d67b773/nova/virt/hardware.py#L875 This works normally because of loop variables in Python are available outside of the scope of the loop: >>> for x in range(5): ... pass ... >>> print(x) 4 and because there's usually something in sibling_sets. However, this is presumably failing for this user because there are no free cores at all on the given host. This is likely the race condition between the nova-scheduler and nova-compute services. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1821733/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp