Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
Great, Jon. Thanks for your reply. I am looking forward to your report.
Cheers,
Boxiang
On 10/23/2018 22:01，Jon Bernard wrote：
* melanie witt wrote:
On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
I created a new vm and a new volume with type 'ceph'[So that the volume
will be created on one of two hosts. I assume that the volume created on
host dev@rbd-1#ceph this time]. Next step is to attach the volume to the
vm. At last I want to migrate the volume from host dev@rbd-1#ceph to
host dev@rbd-2#ceph, but it failed with the exception
'NotImplementedError(_("Swap only supports host devices")'.
So that, my real problem is that is there any work to migrate
volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
in the same ceph cluster?
The difference between the spec[2] with my scope is only one is
*available*(the spec) and another is *in-use*(my scope).
[1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
[2] https://review.openstack.org/#/c/296150
Ah, I think I understand now, thank you for providing all of those details.
And I think you explained it in your first email, that cinder supports
migration of ceph volumes if they are 'available' but not if they are
'in-use'. Apologies that I didn't get your meaning the first time.
I see now the code you were referring to is this [3]:
if volume.status not in ('available', 'retyping', 'maintenance'):
LOG.debug('Only available volumes can be migrated using backend '
'assisted migration. Falling back to generic migration.')
return refuse_to_migrate
So because your volume is not 'available', 'retyping', or 'maintenance',
it's falling back to generic migration, which will end up with an error in
nova because the source_path is not set in the volume config.
Can anyone from the cinder team chime in about whether the ceph volume
migration could be expanded to allow migration of 'in-use' volumes? Is there
a reason not to allow migration of 'in-use' volumes?
Generally speaking, Nova must facilitate the migration of a live (or
in-use) volume. A volume attached to a running instance requires code
in the I/O path to correctly route traffic to the correct location - so
Cinder must refuse (or defer) a migrate operation if the volume is
attached. Until somewhat recently Qemu and Libvirt did not support the
migration to non-block (RBD) targets which is the reason for lack of
support. I believe we now have all of the pieces to perform this
operation successfully, but I suspect it will require a setup with
correct versions of all the related software. I will try to verify this
during the current release cycle and report back.
--
Jon
[3]
https://github.com/openstack/cinder/blob/c42fdc470223d27850627fd4fc9d8cb15f2941f8/cinder/volume/drivers/rbd.py#L1618-L1621
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
On Tue, 23 Oct 2018 10:01:42 -0400, Jon Bernard wrote:
* melanie witt wrote:
On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
I created a new vm and a new volume with type 'ceph'[So that the volume
will be created on one of two hosts. I assume that the volume created on
host dev@rbd-1#ceph this time]. Next step is to attach the volume to the
vm. At last I want to migrate the volume from host dev@rbd-1#ceph to
host dev@rbd-2#ceph, but it failed with the exception
'NotImplementedError(_("Swap only supports host devices")'.
So that, my real problem is that is there any work to migrate
volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
in the same ceph cluster?
The difference between the spec[2] with my scope is only one is
*available*(the spec) and another is *in-use*(my scope).
[1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
[2] https://review.openstack.org/#/c/296150
Ah, I think I understand now, thank you for providing all of those details.
And I think you explained it in your first email, that cinder supports
migration of ceph volumes if they are 'available' but not if they are
'in-use'. Apologies that I didn't get your meaning the first time.
I see now the code you were referring to is this [3]:
if volume.status not in ('available', 'retyping', 'maintenance'):
LOG.debug('Only available volumes can be migrated using backend '
'assisted migration. Falling back to generic migration.')
return refuse_to_migrate
So because your volume is not 'available', 'retyping', or 'maintenance',
it's falling back to generic migration, which will end up with an error in
nova because the source_path is not set in the volume config.
Can anyone from the cinder team chime in about whether the ceph volume
migration could be expanded to allow migration of 'in-use' volumes? Is there
a reason not to allow migration of 'in-use' volumes?
Generally speaking, Nova must facilitate the migration of a live (or
in-use) volume. A volume attached to a running instance requires code
in the I/O path to correctly route traffic to the correct location - so
Cinder must refuse (or defer) a migrate operation if the volume is
attached. Until somewhat recently Qemu and Libvirt did not support the
migration to non-block (RBD) targets which is the reason for lack of
support. I believe we now have all of the pieces to perform this
operation successfully, but I suspect it will require a setup with
correct versions of all the related software. I will try to verify this
during the current release cycle and report back.
OK, thanks for this info, Jon. I'll be interested in your findings.
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
On 10/23/2018 9:01 AM, Jon Bernard wrote:
* melanie witt wrote:
On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
I created a new vm and a new volume with type 'ceph'[So that the volume
will be created on one of two hosts. I assume that the volume created on
host dev@rbd-1#ceph this time]. Next step is to attach the volume to the
vm. At last I want to migrate the volume from host dev@rbd-1#ceph to
host dev@rbd-2#ceph, but it failed with the exception
'NotImplementedError(_("Swap only supports host devices")'.
So that, my real problem is that is there any work to migrate
volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
in the same ceph cluster?
The difference between the spec[2] with my scope is only one is
*available*(the spec) and another is *in-use*(my scope).
[1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
[2] https://review.openstack.org/#/c/296150
Ah, I think I understand now, thank you for providing all of those details.
And I think you explained it in your first email, that cinder supports
migration of ceph volumes if they are 'available' but not if they are
'in-use'. Apologies that I didn't get your meaning the first time.
I see now the code you were referring to is this [3]:
if volume.status not in ('available', 'retyping', 'maintenance'):
LOG.debug('Only available volumes can be migrated using backend '
'assisted migration. Falling back to generic migration.')
return refuse_to_migrate
So because your volume is not 'available', 'retyping', or 'maintenance',
it's falling back to generic migration, which will end up with an error in
nova because the source_path is not set in the volume config.
Can anyone from the cinder team chime in about whether the ceph volume
migration could be expanded to allow migration of 'in-use' volumes? Is there
a reason not to allow migration of 'in-use' volumes?
Generally speaking, Nova must facilitate the migration of a live (or
in-use) volume. A volume attached to a running instance requires code
in the I/O path to correctly route traffic to the correct location - so
Cinder must refuse (or defer) a migrate operation if the volume is
attached. Until somewhat recently Qemu and Libvirt did not support the
migration to non-block (RBD) targets which is the reason for lack of
support. I believe we now have all of the pieces to perform this
operation successfully, but I suspect it will require a setup with
correct versions of all the related software. I will try to verify this
during the current release cycle and report back.
Jon,
Thanks for the explanation and investigation!
Jay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
* melanie witt wrote:
> On Fri, 19 Oct 2018 23:21:01 +0800 (GMT+08:00), Boxiang Zhu wrote:
> >
> > The version of my cinder and nova is Rocky. The scope of the cinder spec[1]
> > is only for available volume migration between two pools from the same
> > ceph cluster.
> > If the volume is in-use status[2], it will call the generic migration
> > function. So that as you
> > describe it, on the nova side, it raises NotImplementedError(_("Swap
> > only supports host devices").
> > The get_config of net volume[3] has not source_path.
>
> Ah, OK, so you're trying to migrate a volume across two separate ceph
> clusters, and that is not supported.
>
> > So does anyone try to succeed to migrate volume(in-use) with ceph
> > backend or is anyone doing something of it?
>
> Hopefully someone can share their experience with trying to migrate volumes
> across separate ceph clusters. I unfortunately don't know anything about it.
If this is the case, then Cinder cannot request a storage-specific
migration which is typically more efficient. The migration will require
a complete copy of each allocated block. Whether the volume is attached
or not will determine who (cinder or nova) will perform the operation.
--
Jon
>
> Best,
> -melanie
>
> > [1] https://review.openstack.org/#/c/296150
> > [2] https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py
> > [3]
> > https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
* melanie witt wrote:
> On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
> > I created a new vm and a new volume with type 'ceph'[So that the volume
> > will be created on one of two hosts. I assume that the volume created on
> > host dev@rbd-1#ceph this time]. Next step is to attach the volume to the
> > vm. At last I want to migrate the volume from host dev@rbd-1#ceph to
> > host dev@rbd-2#ceph, but it failed with the exception
> > 'NotImplementedError(_("Swap only supports host devices")'.
> >
> > So that, my real problem is that is there any work to migrate
> > volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
> > in the same ceph cluster?
> > The difference between the spec[2] with my scope is only one is
> > *available*(the spec) and another is *in-use*(my scope).
> >
> >
> > [1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
> > [2] https://review.openstack.org/#/c/296150
>
> Ah, I think I understand now, thank you for providing all of those details.
> And I think you explained it in your first email, that cinder supports
> migration of ceph volumes if they are 'available' but not if they are
> 'in-use'. Apologies that I didn't get your meaning the first time.
>
> I see now the code you were referring to is this [3]:
>
> if volume.status not in ('available', 'retyping', 'maintenance'):
> LOG.debug('Only available volumes can be migrated using backend '
> 'assisted migration. Falling back to generic migration.')
> return refuse_to_migrate
>
> So because your volume is not 'available', 'retyping', or 'maintenance',
> it's falling back to generic migration, which will end up with an error in
> nova because the source_path is not set in the volume config.
>
> Can anyone from the cinder team chime in about whether the ceph volume
> migration could be expanded to allow migration of 'in-use' volumes? Is there
> a reason not to allow migration of 'in-use' volumes?
Generally speaking, Nova must facilitate the migration of a live (or
in-use) volume. A volume attached to a running instance requires code
in the I/O path to correctly route traffic to the correct location - so
Cinder must refuse (or defer) a migrate operation if the volume is
attached. Until somewhat recently Qemu and Libvirt did not support the
migration to non-block (RBD) targets which is the reason for lack of
support. I believe we now have all of the pieces to perform this
operation successfully, but I suspect it will require a setup with
correct versions of all the related software. I will try to verify this
during the current release cycle and report back.
--
Jon
>
> [3]
> https://github.com/openstack/cinder/blob/c42fdc470223d27850627fd4fc9d8cb15f2941f8/cinder/volume/drivers/rbd.py#L1618-L1621
>
> Cheers,
> -melanie
>
>
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
On Mon, 22 Oct 2018 11:45:55 +0800 (GMT+08:00), Boxiang Zhu wrote:
I created a new vm and a new volume with type 'ceph'[So that the volume
will be created on one of two hosts. I assume that the volume created on
host dev@rbd-1#ceph this time]. Next step is to attach the volume to the
vm. At last I want to migrate the volume from host dev@rbd-1#ceph to
host dev@rbd-2#ceph, but it failed with the exception
'NotImplementedError(_("Swap only supports host devices")'.
So that, my real problem is that is there any work to migrate
volume(*in-use*)(*ceph rbd*) from one host(pool) to another host(pool)
in the same ceph cluster?
The difference between the spec[2] with my scope is only one is
*available*(the spec) and another is *in-use*(my scope).
[1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
[2] https://review.openstack.org/#/c/296150
Ah, I think I understand now, thank you for providing all of those
details. And I think you explained it in your first email, that cinder
supports migration of ceph volumes if they are 'available' but not if
they are 'in-use'. Apologies that I didn't get your meaning the first time.
I see now the code you were referring to is this [3]:
if volume.status not in ('available', 'retyping', 'maintenance'):
LOG.debug('Only available volumes can be migrated using backend '
'assisted migration. Falling back to generic migration.')
return refuse_to_migrate
So because your volume is not 'available', 'retyping', or 'maintenance',
it's falling back to generic migration, which will end up with an error
in nova because the source_path is not set in the volume config.
Can anyone from the cinder team chime in about whether the ceph volume
migration could be expanded to allow migration of 'in-use' volumes? Is
there a reason not to allow migration of 'in-use' volumes?
[3]
https://github.com/openstack/cinder/blob/c42fdc470223d27850627fd4fc9d8cb15f2941f8/cinder/volume/drivers/rbd.py#L1618-L1621
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
Jay and Melanie, It's my fault to let you misunderstand the problem. I should
describe my problem more clearly. My problem is not to migrate volumes between
two ceph clusters.
I have two clusters, one is openstack cluster(allinone env, hostname is dev)
and another is ceph cluster. Omit the integrated configurations for openstack
and ceph.[1] The special config of cinder.conf is as followed:
[DEFAULT]
enabled_backends = rbd-1,rbd-2
..
[rbd-1]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph
rbd_pool = volumes001
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = true
rbd_max_clone_depth = 2
rbd_store_chunk_size = 4
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid = 86d3922a-b471-4dc1-bb89-b46ab7024e81
[rbd-2]
volume_driver = cinder.volume.drivers.rbd.RBDDriver
volume_backend_name = ceph
rbd_pool = volumes002
rbd_ceph_conf = /etc/ceph/ceph.conf
rbd_flatten_volume_from_snapshot = true
rbd_max_clone_depth = 2
rbd_store_chunk_size = 4
rados_connect_timeout = 5
rbd_user = cinder
rbd_secret_uuid = 86d3922a-b471-4dc1-bb89-b46ab7024e81
There will be two hosts named dev@rbd-1#ceph and dev@rbd-2#ceph.
Then I create a volume type named 'ceph' with the command 'cinder type-create
ceph' and add extra_spec 'volume_backend_name=ceph' for it with the command
'cinder type-key set volume_backend_name=ceph'.
I created a new vm and a new volume with type 'ceph'[So that the volume will be
created on one of two hosts. I assume that the volume created on host
dev@rbd-1#ceph this time]. Next step is to attach the volume to the vm. At last
I want to migrate the volume from host dev@rbd-1#ceph to host dev@rbd-2#ceph,
but it failed with the exception 'NotImplementedError(_("Swap only supports
host devices")'.
So that, my real problem is that is there any work to migrate
volume(in-use)(ceph rbd) from one host(pool) to another host(pool) in the same
ceph cluster?
The difference between the spec[2] with my scope is only one is available(the
spec) and another is in-use(my scope).
[1] http://docs.ceph.com/docs/master/rbd/rbd-openstack/
[2] https://review.openstack.org/#/c/296150
Cheers,
Boxiang
On 10/21/2018 23:19，Jay S. Bryant wrote：
Boxiang,
I have not herd any discussion of extending this functionality for Ceph to work
between different Ceph Clusters. I wasn't aware, however, that the existing
spec was limited to one Ceph cluster. So, that is good to know.
I would recommend reaching out to Jon Bernard or Eric Harney for guidance on
how to proceed. They work closely with the Ceph driver and could provide
insight.
Jay
On 10/19/2018 10:21 AM, Boxiang Zhu wrote:
Hi melanie, thanks for your reply.
The version of my cinder and nova is Rocky. The scope of the cinder spec[1]
is only for available volume migration between two pools from the same ceph
cluster.
If the volume is in-use status[2], it will call the generic migration function.
So that as you
describe it, on the nova side, it raises NotImplementedError(_("Swap only
supports host devices").
The get_config of net volume[3] has not source_path.
So does anyone try to succeed to migrate volume(in-use) with ceph backend or is
anyone doing something of it?
[1] https://review.openstack.org/#/c/296150
[2] https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py
[3]
https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101
Cheers,
Boxiang
On 10/19/2018 22:39，melanie witt wrote：
On Fri, 19 Oct 2018 11:33:52 +0800 (GMT+08:00), Boxiang Zhu wrote:
When I use the LVM backend to create the volume, then attach it to a vm.
I can migrate the volume(in-use) from one host to another. The nova
libvirt will call the 'rebase' to finish it. But if using ceph backend,
it raises exception 'Swap only supports host devices'. So now it does
not support to migrate volume(in-use). Does anyone do this work now? Or
Is there any way to let me migrate volume(in-use) with ceph backend?
What version of cinder and nova are you using?
I found this question/answer on ask.openstack.org:
https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/
and it looks like there was some work done on the cinder side [1] to
enable migration of in-use volumes with ceph semi-recently (Queens).
On the nova side, the code looks for the source_path in the volume
config, and if there is not one present, it raises
NotImplementedError(_("Swap only supports host devices"). So in your
environment, the volume configs must be missing a source_path.
If you are using at least Queens version, then there must be something
additional missing that we would need to do to make the migration work.
[1] https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
Boxiang,
I have not herd any discussion of extending this functionality for Ceph
to work between different Ceph Clusters. I wasn't aware, however, that
the existing spec was limited to one Ceph cluster. So, that is good to know.
I would recommend reaching out to Jon Bernard or Eric Harney for
guidance on how to proceed. They work closely with the Ceph driver and
could provide insight.
Jay
On 10/19/2018 10:21 AM, Boxiang Zhu wrote:
Hi melanie, thanks for your reply.
The version of my cinder and nova is Rocky. The scope of the cinder
spec[1]
is only for available volume migration between two pools from the same
ceph cluster.
If the volume is in-use status[2], it will call the generic migration
function. So that as you
describe it, on the nova side, it raises NotImplementedError(_("Swap
only supports host devices").
The get_config of net volume[3] has not source_path.
So does anyone try to succeed to migrate volume(in-use) with ceph
backend or is anyone doing something of it?
[1] https://review.openstack.org/#/c/296150
[2]
https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py
[3]
https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101
Cheers,
Boxiang
On 10/19/2018 22:39，melanie witt
wrote：
On Fri, 19 Oct 2018 11:33:52 +0800 (GMT+08:00), Boxiang Zhu wrote:
When I use the LVM backend to create the volume, then attach
it to a vm.
I can migrate the volume(in-use) from one host to another. The
nova
libvirt will call the 'rebase' to finish it. But if using ceph
backend,
it raises exception 'Swap only supports host devices'. So now
it does
not support to migrate volume(in-use). Does anyone do this
work now? Or
Is there any way to let me migrate volume(in-use) with ceph
backend?
What version of cinder and nova are you using?
I found this question/answer on ask.openstack.org:
https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/
and it looks like there was some work done on the cinder side [1] to
enable migration of in-use volumes with ceph semi-recently (Queens).
On the nova side, the code looks for the source_path in the volume
config, and if there is not one present, it raises
NotImplementedError(_("Swap only supports host devices"). So in your
environment, the volume configs must be missing a source_path.
If you are using at least Queens version, then there must be
something
additional missing that we would need to do to make the migration
work.
[1] https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
On Fri, 19 Oct 2018 23:21:01 +0800 (GMT+08:00), Boxiang Zhu wrote:
The version of my cinder and nova is Rocky. The scope of the cinder spec[1]
is only for available volume migration between two pools from the same
ceph cluster.
If the volume is in-use status[2], it will call the generic migration
function. So that as you
describe it, on the nova side, it raises NotImplementedError(_("Swap
only supports host devices").
The get_config of net volume[3] has not source_path.
Ah, OK, so you're trying to migrate a volume across two separate ceph
clusters, and that is not supported.
So does anyone try to succeed to migrate volume(in-use) with ceph
backend or is anyone doing something of it?
Hopefully someone can share their experience with trying to migrate
volumes across separate ceph clusters. I unfortunately don't know
anything about it.
Best,
-melanie
[1] https://review.openstack.org/#/c/296150
[2] https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py
[3]
https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
Hi melanie, thanks for your reply.
The version of my cinder and nova is Rocky. The scope of the cinder spec[1]
is only for available volume migration between two pools from the same ceph
cluster.
If the volume is in-use status[2], it will call the generic migration function.
So that as you
describe it, on the nova side, it raises NotImplementedError(_("Swap only
supports host devices").
The get_config of net volume[3] has not source_path.
So does anyone try to succeed to migrate volume(in-use) with ceph backend or is
anyone doing something of it?
[1] https://review.openstack.org/#/c/296150
[2] https://review.openstack.org/#/c/256091/23/cinder/volume/drivers/rbd.py
[3]
https://github.com/openstack/nova/blob/stable/rocky/nova/virt/libvirt/volume/net.py#L101
Cheers,
Boxiang
On 10/19/2018 22:39，melanie witt wrote：
On Fri, 19 Oct 2018 11:33:52 +0800 (GMT+08:00), Boxiang Zhu wrote:
When I use the LVM backend to create the volume, then attach it to a vm.
I can migrate the volume(in-use) from one host to another. The nova
libvirt will call the 'rebase' to finish it. But if using ceph backend,
it raises exception 'Swap only supports host devices'. So now it does
not support to migrate volume(in-use). Does anyone do this work now? Or
Is there any way to let me migrate volume(in-use) with ceph backend?
What version of cinder and nova are you using?
I found this question/answer on ask.openstack.org:
https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/
and it looks like there was some work done on the cinder side [1] to
enable migration of in-use volumes with ceph semi-recently (Queens).
On the nova side, the code looks for the source_path in the volume
config, and if there is not one present, it raises
NotImplementedError(_("Swap only supports host devices"). So in your
environment, the volume configs must be missing a source_path.
If you are using at least Queens version, then there must be something
additional missing that we would need to do to make the migration work.
[1] https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] [nova] Problem of Volume(in-use) Live Migration with ceph backend
On Fri, 19 Oct 2018 11:33:52 +0800 (GMT+08:00), Boxiang Zhu wrote:
When I use the LVM backend to create the volume, then attach it to a vm.
I can migrate the volume(in-use) from one host to another. The nova
libvirt will call the 'rebase' to finish it. But if using ceph backend,
it raises exception 'Swap only supports host devices'. So now it does
not support to migrate volume(in-use). Does anyone do this work now? Or
Is there any way to let me migrate volume(in-use) with ceph backend?
What version of cinder and nova are you using?
I found this question/answer on ask.openstack.org:
https://ask.openstack.org/en/question/112954/volume-migration-fails-notimplementederror-swap-only-supports-host-devices/
and it looks like there was some work done on the cinder side [1] to
enable migration of in-use volumes with ceph semi-recently (Queens).
On the nova side, the code looks for the source_path in the volume
config, and if there is not one present, it raises
NotImplementedError(_("Swap only supports host devices"). So in your
environment, the volume configs must be missing a source_path.
If you are using at least Queens version, then there must be something
additional missing that we would need to do to make the migration work.
[1] https://blueprints.launchpad.net/cinder/+spec/ceph-volume-migrate
Cheers,
-melanie
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
