Public bug reported: Description =========== More specifically the following tempest test in master fails: tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server
Steps to reproduce ================== * Install Devstack from master * Run tempest test `tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server` Expected result =============== The test succeeds. Actual result ============= What happened instead of the expected result? How did the issue look like? Environment =========== 1. Git 1858cf18b940b3636e54eb5aafaf4050bdd02939 (master). So essentially this: https://review.opendev.org/c/openstack/nova/+/909474 As instance creation is impossible without that patch. 2. Which hypervisor did you use? What's the version of that? vmwareapi (VSphere 7.0.3 & ESXi 7.0.3) 2. Which storage type did you use? vmdk on NFS 4.1 3. Which networking type did you use? networking-nsx-t (https://github.com/sapcc/networking-nsx-t) Logs & Configs ============== Can be found here: http://openstack-ci- logs.global.cloud.sap/openstack/nova/1858cf18b940b3636e54eb5aafaf4050bdd02939/index.html The critical exception for this bug report is (abbreviated and reformatted for clarity): ---- req-7aa5ded6-ea97-4010-93c8-9e39389cbfe0 tempest-ServerActionsTestOtherA-839537081 [ 865.017199] env[58735]: ERROR nova.compute.manager [instance: b4d9131c-fc91-4fd4-813b-13b4bdfe1647] Traceback (most recent call last): File "/opt/stack/nova/nova/compute/manager.py", line 10856, in _error_out_instance_on_exception yield File "/opt/stack/nova/nova/compute/manager.py", line 6096, in _resize_instance disk_info = self.driver.migrate_disk_and_power_off( File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 263, in migrate_disk_and_power_off return self._vmops.migrate_disk_and_power_off(context, instance, File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1467, in migrate_disk_and_power_off self._resize_disk(instance, vm_ref, vmdk, flavor) File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1398, in _resize_disk self._volumeops.detach_disk_from_vm(vm_ref, instance, vmdk.device) File "/opt/stack/nova/nova/virt/vmwareapi/volumeops.py", line 121, in detach_disk_from_vm disk_key = device.key AttributeError: 'NoneType' object has no attribute 'key' --- The bug is actually in the function `nova.virt.vmwareapi.vm_util.get_vmdk_info` here: https://opendev.org/openstack/nova/src/branch/master/nova/virt/vmwareapi/vm_util.py#L690 The code works with the assumption, that the root-disk is named as the instance. This assumption breaks in several cases, but most for this test-case, the root volume is actually a cinder volume. It will also break when the the disk gets migrated to another datastore, either through a live-migration with no shared storage, or simply automatically with SDRS.. I have an alternative implementation here: https://github.com/sapcc/nova/blob/stable/xena-m3/nova/virt/vmwareapi/vm_util.py#L997-L1034 I'll provide a bug fix from it. ** Affects: nova Importance: Undecided Status: New -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/2055411 Title: Nova VMwareapi Resize of Volume Backed server fails Status in OpenStack Compute (nova): New Bug description: Description =========== More specifically the following tempest test in master fails: tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server Steps to reproduce ================== * Install Devstack from master * Run tempest test `tempest.api.compute.servers.test_server_actions.ServerActionsV293TestJSON.test_rebuild_volume_backed_server` Expected result =============== The test succeeds. Actual result ============= What happened instead of the expected result? How did the issue look like? Environment =========== 1. Git 1858cf18b940b3636e54eb5aafaf4050bdd02939 (master). So essentially this: https://review.opendev.org/c/openstack/nova/+/909474 As instance creation is impossible without that patch. 2. Which hypervisor did you use? What's the version of that? vmwareapi (VSphere 7.0.3 & ESXi 7.0.3) 2. Which storage type did you use? vmdk on NFS 4.1 3. Which networking type did you use? networking-nsx-t (https://github.com/sapcc/networking-nsx-t) Logs & Configs ============== Can be found here: http://openstack-ci- logs.global.cloud.sap/openstack/nova/1858cf18b940b3636e54eb5aafaf4050bdd02939/index.html The critical exception for this bug report is (abbreviated and reformatted for clarity): ---- req-7aa5ded6-ea97-4010-93c8-9e39389cbfe0 tempest-ServerActionsTestOtherA-839537081 [ 865.017199] env[58735]: ERROR nova.compute.manager [instance: b4d9131c-fc91-4fd4-813b-13b4bdfe1647] Traceback (most recent call last): File "/opt/stack/nova/nova/compute/manager.py", line 10856, in _error_out_instance_on_exception yield File "/opt/stack/nova/nova/compute/manager.py", line 6096, in _resize_instance disk_info = self.driver.migrate_disk_and_power_off( File "/opt/stack/nova/nova/virt/vmwareapi/driver.py", line 263, in migrate_disk_and_power_off return self._vmops.migrate_disk_and_power_off(context, instance, File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1467, in migrate_disk_and_power_off self._resize_disk(instance, vm_ref, vmdk, flavor) File "/opt/stack/nova/nova/virt/vmwareapi/vmops.py", line 1398, in _resize_disk self._volumeops.detach_disk_from_vm(vm_ref, instance, vmdk.device) File "/opt/stack/nova/nova/virt/vmwareapi/volumeops.py", line 121, in detach_disk_from_vm disk_key = device.key AttributeError: 'NoneType' object has no attribute 'key' --- The bug is actually in the function `nova.virt.vmwareapi.vm_util.get_vmdk_info` here: https://opendev.org/openstack/nova/src/branch/master/nova/virt/vmwareapi/vm_util.py#L690 The code works with the assumption, that the root-disk is named as the instance. This assumption breaks in several cases, but most for this test-case, the root volume is actually a cinder volume. It will also break when the the disk gets migrated to another datastore, either through a live-migration with no shared storage, or simply automatically with SDRS.. I have an alternative implementation here: https://github.com/sapcc/nova/blob/stable/xena-m3/nova/virt/vmwareapi/vm_util.py#L997-L1034 I'll provide a bug fix from it. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/2055411/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp