[Yahoo-eng-team] [Bug 1860990] Re: RBD image backend tries to flatten images even if they are already flat
** Changed in: nova/train Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1860990 Title: RBD image backend tries to flatten images even if they are already flat Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) queens series: Fix Committed Status in OpenStack Compute (nova) rocky series: Fix Committed Status in OpenStack Compute (nova) stein series: Fix Released Status in OpenStack Compute (nova) train series: Fix Released Bug description: When [DEFAULT]show_multiple_locations option is not set in glance, and both glance and nova use ceph as their backend, with properly configured accesses, nova will fail with the following exception: 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [req-8021fd76-d5ab-4a9b-bd17-f5eb4d4faf62 0e96a04f360644818632b7e46fe8d3e7 ac01daacc7424a40b8b464a163902dcb - default default] [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Instance failed to spawn: rbd.InvalidArgument: [errno 22] error flattening b'fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6_disk' 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Traceback (most recent call last): 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/compute/manager.py", line 5757, in _unshelve_instance 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3457, in spawn 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3832, in _create_image 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3923, in _create_and_inject_local_root 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] instance, size, fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 9267, in _try_fetch_image_cache 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] image.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 983, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] self.driver.flatten(self.rbd_name, pool=self.driver.pool) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 290, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] vol.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] rv = execute(f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6]
[Yahoo-eng-team] [Bug 1860990] Re: RBD image backend tries to flatten images even if they are already flat
** Changed in: nova/stein Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1860990 Title: RBD image backend tries to flatten images even if they are already flat Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) queens series: Fix Committed Status in OpenStack Compute (nova) rocky series: Fix Committed Status in OpenStack Compute (nova) stein series: Fix Released Status in OpenStack Compute (nova) train series: Fix Committed Bug description: When [DEFAULT]show_multiple_locations option is not set in glance, and both glance and nova use ceph as their backend, with properly configured accesses, nova will fail with the following exception: 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [req-8021fd76-d5ab-4a9b-bd17-f5eb4d4faf62 0e96a04f360644818632b7e46fe8d3e7 ac01daacc7424a40b8b464a163902dcb - default default] [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Instance failed to spawn: rbd.InvalidArgument: [errno 22] error flattening b'fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6_disk' 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Traceback (most recent call last): 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/compute/manager.py", line 5757, in _unshelve_instance 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3457, in spawn 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3832, in _create_image 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3923, in _create_and_inject_local_root 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] instance, size, fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 9267, in _try_fetch_image_cache 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] image.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 983, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] self.driver.flatten(self.rbd_name, pool=self.driver.pool) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 290, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] vol.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] rv = execute(f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance:
[Yahoo-eng-team] [Bug 1860990] Re: RBD image backend tries to flatten images even if they are already flat
** Also affects: nova/stein Importance: Undecided Status: New ** Also affects: nova/queens Importance: Undecided Status: New ** Also affects: nova/rocky Importance: Undecided Status: New ** Changed in: nova/queens Importance: Undecided => Medium ** Changed in: nova/queens Status: New => In Progress ** Changed in: nova/queens Assignee: (unassigned) => Lee Yarwood (lyarwood) ** Changed in: nova/stein Importance: Undecided => Medium ** Changed in: nova/stein Status: New => In Progress ** Changed in: nova/stein Assignee: (unassigned) => Lee Yarwood (lyarwood) ** Changed in: nova/rocky Importance: Undecided => Medium ** Changed in: nova/rocky Status: New => In Progress ** Changed in: nova/rocky Assignee: (unassigned) => Lee Yarwood (lyarwood) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1860990 Title: RBD image backend tries to flatten images even if they are already flat Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) queens series: In Progress Status in OpenStack Compute (nova) rocky series: In Progress Status in OpenStack Compute (nova) stein series: In Progress Status in OpenStack Compute (nova) train series: In Progress Bug description: When [DEFAULT]show_multiple_locations option is not set in glance, and both glance and nova use ceph as their backend, with properly configured accesses, nova will fail with the following exception: 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [req-8021fd76-d5ab-4a9b-bd17-f5eb4d4faf62 0e96a04f360644818632b7e46fe8d3e7 ac01daacc7424a40b8b464a163902dcb - default default] [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Instance failed to spawn: rbd.InvalidArgument: [errno 22] error flattening b'fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6_disk' 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Traceback (most recent call last): 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/compute/manager.py", line 5757, in _unshelve_instance 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3457, in spawn 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3832, in _create_image 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3923, in _create_and_inject_local_root 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] instance, size, fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 9267, in _try_fetch_image_cache 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] image.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 983, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] self.driver.flatten(self.rbd_name, pool=self.driver.pool) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 290, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] vol.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance:
[Yahoo-eng-team] [Bug 1860990] Re: RBD image backend tries to flatten images even if they are already flat
** Also affects: nova/train Importance: Undecided Status: New ** Changed in: nova/train Importance: Undecided => Medium ** Changed in: nova/train Status: New => In Progress ** Changed in: nova/train Assignee: (unassigned) => Lee Yarwood (lyarwood) -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1860990 Title: RBD image backend tries to flatten images even if they are already flat Status in OpenStack Compute (nova): Fix Released Status in OpenStack Compute (nova) train series: In Progress Bug description: When [DEFAULT]show_multiple_locations option is not set in glance, and both glance and nova use ceph as their backend, with properly configured accesses, nova will fail with the following exception: 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [req-8021fd76-d5ab-4a9b-bd17-f5eb4d4faf62 0e96a04f360644818632b7e46fe8d3e7 ac01daacc7424a40b8b464a163902dcb - default default] [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Instance failed to spawn: rbd.InvalidArgument: [errno 22] error flattening b'fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6_disk' 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Traceback (most recent call last): 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/compute/manager.py", line 5757, in _unshelve_instance 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3457, in spawn 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3832, in _create_image 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3923, in _create_and_inject_local_root 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] instance, size, fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 9267, in _try_fetch_image_cache 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] image.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 983, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] self.driver.flatten(self.rbd_name, pool=self.driver.pool) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 290, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] vol.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 148, in proxy_call 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] rv = execute(f, *args, **kwargs) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 129, in execute 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance:
[Yahoo-eng-team] [Bug 1860990] Re: RBD image backend tries to flatten images even if they are already flat
Reviewed: https://review.opendev.org/704330 Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=65825ebfbd58920adac5e8594891eec8e9cec41f Submitter: Zuul Branch:master commit 65825ebfbd58920adac5e8594891eec8e9cec41f Author: Vladyslav Drok Date: Mon Jan 27 15:31:53 2020 +0100 Make RBD imagebackend flatten method idempotent If glance and nova are both configured with RBD backend, but glance does not return location information from the API, nova will fail to clone the image from glance pool and will download it from the API. In this case, image will be already flat, and subsequent flatten call will fail. This commit makes flatten call idempotent, so that it ignores already flat images by catching ImageUnacceptable when requesting parent info from ceph. Closes-Bug: 1860990 Change-Id: Ia6c184c31a980e4728b7309b2afaec4d9f494ac3 ** Changed in: nova Status: In Progress => Fix Released -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1860990 Title: RBD image backend tries to flatten images even if they are already flat Status in OpenStack Compute (nova): Fix Released Bug description: When [DEFAULT]show_multiple_locations option is not set in glance, and both glance and nova use ceph as their backend, with properly configured accesses, nova will fail with the following exception: 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [req-8021fd76-d5ab-4a9b-bd17-f5eb4d4faf62 0e96a04f360644818632b7e46fe8d3e7 ac01daacc7424a40b8b464a163902dcb - default default] [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Instance failed to spawn: rbd.InvalidArgument: [errno 22] error flattening b'fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6_disk' 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] Traceback (most recent call last): 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/compute/manager.py", line 5757, in _unshelve_instance 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3457, in spawn 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] block_device_info=block_device_info) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3832, in _create_image 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 3923, in _create_and_inject_local_root 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] instance, size, fallback_from_host) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 9267, in _try_fetch_image_cache 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] image.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/imagebackend.py", line 983, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] self.driver.flatten(self.rbd_name, pool=self.driver.pool) 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/nova/virt/libvirt/storage/rbd_utils.py", line 290, in flatten 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] vol.flatten() 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] File "/var/lib/openstack/lib/python3.6/site-packages/eventlet/tpool.py", line 190, in doit 2020-01-23 14:36:43.617 8647 ERROR nova.compute.manager [instance: fa9e4118-1bb1-4d52-a2e1-9f61b0e20dc6] result = proxy_call(self._autowrap, f, *args, **kwargs) 2020-01-23
