Public bug reported:

Env info:
openstack version: rocky
storage back-end: ceph
hypervisor: qemu/KVM

Sample traceback:
==================================
[req-69c94c9a-6ee4-4936-8ce5-9a23b7aea89a 00b865b2a29e47f8b57a62ac624bdfa4 
9edd1f98bf2f47e885f7077a066c83dd - default default] 
[instance: 642ab2df-4dc2-4ca8-9bbd-ab19c72352df] 
Setting instance vm_state to ERROR: OSError: [Errno 39] Directory not empty
Traceback (most recent call last):
  File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 
8333, in _error_out_instance_on_exception
    yield
  File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 
4693, in _resize_instance
    timeout, retry_interval)
  File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
line 8668, in migrate_disk_and_power_off
    shared_storage)
  File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", 
line 220, in __exit__
    self.force_reraise()
  File "/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", 
line 196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
line 8622, in migrate_disk_and_power_off
    os.rename(inst_base, inst_base_resize)
OSError: [Errno 39] Directory not empty


Description:
==================================
1. Executing VM resize error, and the the dir of inst_base_resize has been 
created by `os.rename(inst_base, inst_base_resize)` in the function of 
migrate_disk_and_power_off. The function of `_error_out_instance_on_exception` 
in _resize_instance just catch exceptions and can not rollback dir.

2. Executing command of `openstack server set` to recover VM status to
active.

3. Executing VM resize error again, and Exception in Sample traceback.
The operation of `os.rename(inst_base, inst_base_resize)` failed because
of the dir of inst_base_resize has console.log.

4. Whether or not execute `self._cleanup_remote_migration` before
`os.rename(inst_base, inst_base_resize)`?  Is there any methods to
optimize exception handling in the function of _resize_instance ?

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1919401

Title:
  the function of _resize_instance lack of exception handling

Status in OpenStack Compute (nova):
  New

Bug description:
  Env info:
  openstack version: rocky
  storage back-end: ceph
  hypervisor: qemu/KVM

  Sample traceback:
  ==================================
  [req-69c94c9a-6ee4-4936-8ce5-9a23b7aea89a 00b865b2a29e47f8b57a62ac624bdfa4 
9edd1f98bf2f47e885f7077a066c83dd - default default] 
  [instance: 642ab2df-4dc2-4ca8-9bbd-ab19c72352df] 
  Setting instance vm_state to ERROR: OSError: [Errno 39] Directory not empty
  Traceback (most recent call last):
    File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 
8333, in _error_out_instance_on_exception
      yield
    File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/compute/manager.py", line 
4693, in _resize_instance
      timeout, retry_interval)
    File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
line 8668, in migrate_disk_and_power_off
      shared_storage)
    File 
"/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 
220, in __exit__
      self.force_reraise()
    File 
"/var/lib/openstack/lib/python2.7/site-packages/oslo_utils/excutils.py", line 
196, in force_reraise
      six.reraise(self.type_, self.value, self.tb)
    File 
"/var/lib/openstack/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", 
line 8622, in migrate_disk_and_power_off
      os.rename(inst_base, inst_base_resize)
  OSError: [Errno 39] Directory not empty

  
  Description:
  ==================================
  1. Executing VM resize error, and the the dir of inst_base_resize has been 
created by `os.rename(inst_base, inst_base_resize)` in the function of 
migrate_disk_and_power_off. The function of `_error_out_instance_on_exception` 
in _resize_instance just catch exceptions and can not rollback dir.

  2. Executing command of `openstack server set` to recover VM status to
  active.

  3. Executing VM resize error again, and Exception in Sample traceback.
  The operation of `os.rename(inst_base, inst_base_resize)` failed
  because of the dir of inst_base_resize has console.log.

  4. Whether or not execute `self._cleanup_remote_migration` before
  `os.rename(inst_base, inst_base_resize)`?  Is there any methods to
  optimize exception handling in the function of _resize_instance ?

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1919401/+subscriptions

-- 
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to     : yahoo-eng-team@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help   : https://help.launchpad.net/ListHelp

Reply via email to