[ceph-users] rbd and exclusive lock feature
Hi guys, when creating an rbd with feature "exclusive-lock" (and object-map, fast-diff, ..), do I have to pass any special arguments to qemu to activate it? How does this feature work with resize, snapshot creation, etc.? Form my work on ceph-ruby I know you have to call "rbd_open" and then pass the returned handle to "rbd_resize", "rbd_snap_create" etc. However with "exclusive-lock" I assume rbd_open will fail (does it?) if it's currently used by for example a qemu client. So there's no way to create snapshots while the image is in use? Or do I miss anything here? Cheers Corin ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd and exclusive lock feature
The naming of the "exclusive-lock" feature probably implies too much compared to what it actually does. In reality, when you enable the "exclusive-lock" feature, only one RBD client is able to modify the image while the lock is held. However, that won't stop other RBD clients from *requesting* that maintenance operations be performed on the image (e.g. snapshot, resize). Behind the scenes, librbd will detect that another client currently owns the lock and will proxy its request over to the current watch owner. This ensures that we only have one client modifying the image while at the same time not crippling other use cases. librbd also supports cooperative exclusive lock transfer, which is used in the case of qemu VM migrations where the image needs to be opened R/W by two clients at the same time. -- Jason Dillaman - Original Message - > From: "Corin Langosch" > To: ceph-users@lists.ceph.com > Sent: Tuesday, September 22, 2015 8:11:38 AM > Subject: [ceph-users] rbd and exclusive lock feature > > Hi guys, > > when creating an rbd with feature "exclusive-lock" (and object-map, > fast-diff, ..), do I have to pass any special > arguments to qemu to activate it? > > How does this feature work with resize, snapshot creation, etc.? Form my work > on ceph-ruby I know you have to call > "rbd_open" and then pass the returned handle to "rbd_resize", > "rbd_snap_create" etc. However with "exclusive-lock" I > assume rbd_open will fail (does it?) if it's currently used by for example a > qemu client. So there's no way to create > snapshots while the image is in use? Or do I miss anything here? > > Cheers > Corin > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd and exclusive lock feature
> when you enable the "exclusive-lock" feature, only one RBD client is able to > modify the image while the lock is held. means rbd_lock_exclusive > However, that won't stop other RBD clients from *requesting* that maintenance > operations be performed on the image (e.g. snapshot, resize). means rbd_lock_shared Correct? - Shinobu - Original Message - From: "Jason Dillaman" To: "Corin Langosch" Cc: ceph-users@lists.ceph.com Sent: Tuesday, September 22, 2015 9:37:56 PM Subject: Re: [ceph-users] rbd and exclusive lock feature The naming of the "exclusive-lock" feature probably implies too much compared to what it actually does. In reality, when you enable the "exclusive-lock" feature, only one RBD client is able to modify the image while the lock is held. However, that won't stop other RBD clients from *requesting* that maintenance operations be performed on the image (e.g. snapshot, resize). Behind the scenes, librbd will detect that another client currently owns the lock and will proxy its request over to the current watch owner. This ensures that we only have one client modifying the image while at the same time not crippling other use cases. librbd also supports cooperative exclusive lock transfer, which is used in the case of qemu VM migrations where the image needs to be opened R/W by two clients at the same time. -- Jason Dillaman - Original Message - > From: "Corin Langosch" > To: ceph-users@lists.ceph.com > Sent: Tuesday, September 22, 2015 8:11:38 AM > Subject: [ceph-users] rbd and exclusive lock feature > > Hi guys, > > when creating an rbd with feature "exclusive-lock" (and object-map, > fast-diff, ..), do I have to pass any special > arguments to qemu to activate it? > > How does this feature work with resize, snapshot creation, etc.? Form my work > on ceph-ruby I know you have to call > "rbd_open" and then pass the returned handle to "rbd_resize", > "rbd_snap_create" etc. However with "exclusive-lock" I > assume rbd_open will fail (does it?) if it's currently used by for example a > qemu client. So there's no way to create > snapshots while the image is in use? Or do I miss anything here? > > Cheers > Corin > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd and exclusive lock feature
> > when you enable the "exclusive-lock" feature, only one RBD client is able > > to modify the image while the lock is held. > > means > > rbd_lock_exclusive > > > However, that won't stop other RBD clients from *requesting* that > > maintenance operations be performed on the image (e.g. snapshot, resize). > > means > > rbd_lock_shared > > Correct? > There are two RBD locking features: the legacy RBD CLI lock and the new exclusive-lock feature bit. The legacy lock was managed via the 'rbd lock add/remove' commands and was purely advisory / not enforced within librbd. A possible use case for such locking would be for failover where prior to using the image you lock it. If someone else owns the lock, you could blacklist that client, break their lock, and take the lock for yourself. The new exclusive-lock feature is managed via 'rbd feature enable/disable' commands and does ensure that only the current lock owner can manipulate the RBD image. It was introduced to support the RBD object map feature (which can track which backing RADOS objects are in-use in order to activate some optimizations) and the future RBD journal and mirroring features. -- Jason Dillaman ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
