What’s the config on PURE?
> On May 29, 2018, at 12:03 PM, Steven D. Searles <ssear...@zimcom.net> wrote:
>
> Hello everyone, I am seeing a strange issue with cinder block live migration
> and libvirt and looking for some assistance.
>
> Environment: Openstack Pike
> OS: Ubuntu 16.04LTS
> Cinder FC Driver: Pure Storage
> Cinder FC Driver: Dell Compellent
> libvirtd (libvirt) 3.6.0
> Cinder-volume 2:11.1.0-0ubuntu1~cloud0
>
> I am trying to migrate some volumes from our older Dell Compellent SC storage
> arrays to new Pure Storage arrays. I can create new volumes on both array’s
> via cinder. I can create a volume and boot it on the Pure AFA’s. I can live
> migrate volumes from the Pure array to the Compellent array but not from the
> Compellent to the Pure. The volume is created and data copied to it, it is
> then deleted by Cinder after the failure is logged. I receive the following
> error when going from the Compellent to the Pure Array. I can migrate in
> both directions if the volume is not attached to an instance. Any ideas?
>
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver
> [req-8ab3bc34-d810-4584-8f31-4dd611ed7b98 385c60230b3f49da930dda4d089eda6b
> 723aa12337a44f818b6d1e1a59f16e49 - default default] Failure rebasing volume
> /dev/disk/by-path/pci-0000:03:00.0-fc-0x524a937cddfa5902-lun-2 on vda.:
> libvirtError: Requested operation is not valid: pivot of disk 'vda' requires
> an active copy job
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver Traceback (most
> recent call last):
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1345, in
> _swap_volume
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver
> dev.abort_job(pivot=True)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/guest.py", line 751, in
> abort_job
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver
> self._guest._domain.blockJobAbort(self._disk, flags=flags)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 186, in doit
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver result =
> proxy_call(self._autowrap, f, *args, **kwargs)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 144, in proxy_call
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver rv =
> execute(f, *args, **kwargs)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 125, in execute
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver
> six.reraise(c, e, tb)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 83, in tworker
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver rv =
> meth(*args, **kwargs)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver File
> "/usr/lib/python2.7/dist-packages/libvirt.py", line 766, in blockJobAbort
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver if ret == -1:
> raise libvirtError ('virDomainBlockJobAbort() failed', dom=self)
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver libvirtError:
> Requested operation is not valid: pivot of disk 'vda' requires an active copy
> job
> 2018-05-29 13:08:27.618 3198 ERROR nova.virt.libvirt.driver
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager
> [req-8ab3bc34-d810-4584-8f31-4dd611ed7b98 385c60230b3f49da930dda4d089eda6b
> 723aa12337a44f818b6d1e1a59f16e49 - default default] [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] Failed to swap volume
> bb59023e-d463-44e9-8b1a-a9af495d3d4f for
> ecc059d0-79f9-402e-aea8-8c99004f221d: VolumeRebaseFailed: Volume rebase
> failed: Requested operation is not valid: pivot of disk 'vda' requires an
> active copy job
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] Traceback (most recent call last):
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5114, in
> _swap_volume
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] mountpoint, resize_to)
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1408, in
> swap_volume
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] instance)
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
> __exit__
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] self.force_reraise()
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
> force_reraise
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] six.reraise(self.type_, self.value,
> self.tb)
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1404, in
> swap_volume
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] self._swap_volume(guest, disk_dev,
> conf, resize_to)
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1351, in
> _swap_volume
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] raise
> exception.VolumeRebaseFailed(reason=six.text_type(exc))
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29] VolumeRebaseFailed: Volume rebase
> failed: Requested operation is not valid: pivot of disk 'vda' requires an
> active copy job
> 2018-05-29 13:08:28.719 3198 ERROR nova.compute.manager [instance:
> f834fc03-7e2d-41f4-9307-a1bded3abb29]
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> [req-8ab3bc34-d810-4584-8f31-4dd611ed7b98 385c60230b3f49da930dda4d089eda6b
> 723aa12337a44f818b6d1e1a59f16e49 - default default] Exception during message
> handling: VolumeRebaseFailed: Volume rebase failed: Requested operation is
> not valid: pivot of disk 'vda' requires an active copy job
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server Traceback (most
> recent call last):
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/server.py", line 160, in
> _process_incoming
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server res =
> self.dispatcher.dispatch(message)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
> 213, in dispatch
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server return
> self._do_dispatch(endpoint, method, ctxt, args)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line
> 183, in _do_dispatch
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server result =
> func(ctxt, **new_args)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 76, in
> wrapped
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> function_name, call_dict, binary)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
> __exit__
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> self.force_reraise()
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
> force_reraise
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> six.reraise(self.type_, self.value, self.tb)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/exception_wrapper.py", line 67, in
> wrapped
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server return
> f(self, context, *args, **kw)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 218, in
> decorated_function
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> kwargs['instance'], e, sys.exc_info())
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
> __exit__
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> self.force_reraise()
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
> force_reraise
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> six.reraise(self.type_, self.value, self.tb)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 206, in
> decorated_function
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server return
> function(self, context, *args, **kwargs)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5255, in
> swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> is_cinder_migration)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5154, in
> _swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server context,
> new_attachment_id)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
> __exit__
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> self.force_reraise()
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
> force_reraise
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> six.reraise(self.type_, self.value, self.tb)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 5114, in
> _swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server mountpoint,
> resize_to)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1408, in
> swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server instance)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in
> __exit__
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> self.force_reraise()
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in
> force_reraise
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> six.reraise(self.type_, self.value, self.tb)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1404, in
> swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> self._swap_volume(guest, disk_dev, conf, resize_to)
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1351, in
> _swap_volume
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server raise
> exception.VolumeRebaseFailed(reason=six.text_type(exc))
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
> VolumeRebaseFailed: Volume rebase failed: Requested operation is not valid:
> pivot of disk 'vda' requires an active copy job
> 2018-05-29 13:08:30.417 3198 ERROR oslo_messaging.rpc.server
>
>
> Any help would be greatly appreciated.
>
> Thanks!
>
> — Steve
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
> Post to : openstack@lists.openstack.org
> <mailto:openstack@lists.openstack.org>
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack>
_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to : openstack@lists.openstack.org
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
