Public bug reported: Description ===========
I had one volume, vol-1 attached to two VMs, vm-1 and vm-2, then tried to swap vm-1's volume to another volume vol-2, but failed. The Cinder backend attached the volume to vm successfully. Nova reported below error: Mar 08 01:34:42.984286 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver [None req-1033d1c1-bd3b-4990-a8da-e503d25a041b tempest-TestMultiAttachVolumeSwap-890516006 tempest-TestMultiAttachVolumeSwap-890516006] Failure rebasing volume /dev/sdj on vdb.: libvirt.libvirtError: Requested operation is not valid: pivot of disk 'vdb' requires an active copy job Mar 08 01:34:42.984521 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver Traceback (most recent call last): Mar 08 01:34:42.984846 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1557, in _swap_volume Mar 08 01:34:42.985032 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver dev.abort_job(pivot=True) Mar 08 01:34:42.985214 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 754, in abort_job Mar 08 01:34:42.986537 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver self._guest._domain.blockJobAbort(self._disk, flags=flags) Mar 08 01:34:42.986914 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 190, in doit Mar 08 01:34:42.987246 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver result = proxy_call(self._autowrap, f, *args, **kwargs) Mar 08 01:34:42.987608 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 148, in proxy_call Mar 08 01:34:42.987958 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver rv = execute(f, *args, **kwargs) Mar 08 01:34:42.988276 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 129, in execute Mar 08 01:34:42.988434 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver six.reraise(c, e, tb) Mar 08 01:34:42.988591 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise Mar 08 01:34:42.988746 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver raise value Mar 08 01:34:42.988895 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 83, in tworker Mar 08 01:34:42.989045 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver rv = meth(*args, **kwargs) Mar 08 01:34:42.989194 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/libvirt.py", line 772, in blockJobAbort Mar 08 01:34:42.989366 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver if ret == -1: raise libvirtError ('virDomainBlockJobAbort() failed', dom=self) Mar 08 01:34:42.989516 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver libvirt.libvirtError: Requested operation is not valid: pivot of disk 'vdb' requires an active copy job Steps to reproduce ================== 1. In OpenStack, attach a volume, vol-1 to two vms, vm-1 and vm-2. Volume vol-1 is provisioned on external storage system. 2. Swap vol-1 of vm-1 to another volume, vol-2. Actual results ============== You can see above failure: Failure rebasing volume /dev/sdj (vol-2) on vdb. Expected results ================ vm-1's volume updated to vol-2. Environment =========== OpenStack: master (Stein) devstack libvirt-python: 5.1.0 libvirt-bin: 4.0.0-1ubuntu8.5~cloud0 Logs ==== 1. Full n-cpu log locates: http://publiclogs.emc.com/51/633451/5/check/EMC_Unity_iSCSI/a2d0e05/EMC_Unity_iSCSI/1218/logs/screen-n-cpu.txt.gz Search via request id: req-1033d1c1-bd3b-4990-a8da-e503d25a041b 2. libvirt log is attached. If you need more info, please let me know. ** Affects: nova Importance: Undecided Status: New ** Tags: libvirt ** Attachment added: "libvirtd log" https://bugs.launchpad.net/bugs/1819569/+attachment/5245583/+files/libvirtd.log -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to OpenStack Compute (nova). https://bugs.launchpad.net/bugs/1819569 Title: Swap volumes of vm failed: pivot of disk 'vdb' requires an active copy job Status in OpenStack Compute (nova): New Bug description: Description =========== I had one volume, vol-1 attached to two VMs, vm-1 and vm-2, then tried to swap vm-1's volume to another volume vol-2, but failed. The Cinder backend attached the volume to vm successfully. Nova reported below error: Mar 08 01:34:42.984286 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver [None req-1033d1c1-bd3b-4990-a8da-e503d25a041b tempest-TestMultiAttachVolumeSwap-890516006 tempest-TestMultiAttachVolumeSwap-890516006] Failure rebasing volume /dev/sdj on vdb.: libvirt.libvirtError: Requested operation is not valid: pivot of disk 'vdb' requires an active copy job Mar 08 01:34:42.984521 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver Traceback (most recent call last): Mar 08 01:34:42.984846 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 1557, in _swap_volume Mar 08 01:34:42.985032 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver dev.abort_job(pivot=True) Mar 08 01:34:42.985214 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/opt/stack/new/nova/nova/virt/libvirt/guest.py", line 754, in abort_job Mar 08 01:34:42.986537 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver self._guest._domain.blockJobAbort(self._disk, flags=flags) Mar 08 01:34:42.986914 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 190, in doit Mar 08 01:34:42.987246 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver result = proxy_call(self._autowrap, f, *args, **kwargs) Mar 08 01:34:42.987608 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 148, in proxy_call Mar 08 01:34:42.987958 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver rv = execute(f, *args, **kwargs) Mar 08 01:34:42.988276 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 129, in execute Mar 08 01:34:42.988434 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver six.reraise(c, e, tb) Mar 08 01:34:42.988591 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/six.py", line 693, in reraise Mar 08 01:34:42.988746 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver raise value Mar 08 01:34:42.988895 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/eventlet/tpool.py", line 83, in tworker Mar 08 01:34:42.989045 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver rv = meth(*args, **kwargs) Mar 08 01:34:42.989194 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver File "/usr/local/lib/python3.5/dist-packages/libvirt.py", line 772, in blockJobAbort Mar 08 01:34:42.989366 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver if ret == -1: raise libvirtError ('virDomainBlockJobAbort() failed', dom=self) Mar 08 01:34:42.989516 UnityiSCSISlave nova-compute[19437]: ERROR nova.virt.libvirt.driver libvirt.libvirtError: Requested operation is not valid: pivot of disk 'vdb' requires an active copy job Steps to reproduce ================== 1. In OpenStack, attach a volume, vol-1 to two vms, vm-1 and vm-2. Volume vol-1 is provisioned on external storage system. 2. Swap vol-1 of vm-1 to another volume, vol-2. Actual results ============== You can see above failure: Failure rebasing volume /dev/sdj (vol-2) on vdb. Expected results ================ vm-1's volume updated to vol-2. Environment =========== OpenStack: master (Stein) devstack libvirt-python: 5.1.0 libvirt-bin: 4.0.0-1ubuntu8.5~cloud0 Logs ==== 1. Full n-cpu log locates: http://publiclogs.emc.com/51/633451/5/check/EMC_Unity_iSCSI/a2d0e05/EMC_Unity_iSCSI/1218/logs/screen-n-cpu.txt.gz Search via request id: req-1033d1c1-bd3b-4990-a8da-e503d25a041b 2. libvirt log is attached. If you need more info, please let me know. To manage notifications about this bug go to: https://bugs.launchpad.net/nova/+bug/1819569/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp