Re: [dm-devel] [PATCHv2 0/7] multipathd 'cookie' handling rework
On Mon, May 09, 2016 at 12:52:58PM +0200, Hannes Reinecke wrote: > Hi all, > > here's now the second attempt to fixup the 'cookie' handling > in multipath-tools. > I've removed the patch to pass in a cookie as an argument > for dm_addmap_reload() and dm_simplecmd(). > And I've removed all instances where we had been calling > dm_udev_complete() after failing to set the cookie to a task. > > As usual, reviews and comments are welcome. ACK for the whole set. Thanks for working through all this. -Ben > > Hannes Reinecke (7): > devmapper: do not flush I/O for DM_DEVICE_CREATE > devmapper: do not call dm_udev_complete if dm_task_set_cookie() failed > configure: Rename ACT_RENAME2 to ACT_FORCERENAME > devmapper: wait for udev in dm_simplecmd_noflush() > multipathd: wait for udev in cli_resume() > devmapper: remove 'udev_sync' argument from dm_simplecmd_noflush() > libmultipath: Fixup 'DM_DEVICE_RELOAD' handling > > libmultipath/configure.c | 20 - > libmultipath/configure.h | 2 +- > libmultipath/devmapper.c | 75 > +++ > libmultipath/devmapper.h | 4 +-- > multipathd/cli_handlers.c | 4 +-- > 5 files changed, 60 insertions(+), 45 deletions(-) > > -- > 2.6.6 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] [PATCH 4/7] devmapper: wait for udev in dm_simplecmd_noflush()
On Mon, May 09, 2016 at 12:53:02PM +0200, Hannes Reinecke wrote: > When calling dm_simplecmd_noflush() with udev_flags set we > need to set the 'need_sync' flag otherwise the udev flags > will never be set. The other possibility here would be to temporarily disable udev sync support, and then restore it afterwards. That way we could still pass the flags, but we wouldn't need to wait on these table reloads. Either that or to get libdevmapper to give us a cookie flag that says, "while, I do have sync support enabled, for this one command, I'd like to skip it". But neither of those options has any significant benefit over this solution. ACK. -Ben > > Signed-off-by: Hannes Reinecke> --- > libmultipath/configure.c | 6 -- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/libmultipath/configure.c b/libmultipath/configure.c > index b976da7..9c6904a 100644 > --- a/libmultipath/configure.c > +++ b/libmultipath/configure.c > @@ -628,7 +628,7 @@ domap (struct multipath * mpp, char * params) > r = dm_addmap_reload(mpp, params); > if (r) > r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, > - 0, MPATH_UDEV_RELOAD_FLAG); > + 1, MPATH_UDEV_RELOAD_FLAG); > break; > > case ACT_RESIZE: > @@ -646,7 +646,9 @@ domap (struct multipath * mpp, char * params) > if (r) { > r = dm_addmap_reload(mpp, params); > if (r) > - r = dm_simplecmd_noflush(DM_DEVICE_RESUME, > mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG); > + r = dm_simplecmd_noflush(DM_DEVICE_RESUME, > + mpp->alias, 1, > + > MPATH_UDEV_RELOAD_FLAG); > } > break; > > -- > 2.6.6 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] md: md.c: fix oops in mddev_suspend for raid0
On Mon, May 09 2016 at 11:50am -0400, hei...@redhat.comwrote: > From: Heinz Mauelshagen > > Introduced by upstream commit 70d9798b95562abac005d4ba71d28820f9a201eb > > The raid0 personality does not create mddev->thread as oposed to > other personalities leading to its unconditional access in > mddev_suspend() causing an oops. > > Patch checks for mddev->thread in order to keep the > intention of aforementioned commit. > > Signed-off-by: Heinz Mauelshagen Seems this should be marked with: Fixes: 70d9798b9556 ("MD: warn for potential deadlock") Cc: sta...@vger.kernel.org # 4.5+ > > > --- > drivers/md/md.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/md/md.c b/drivers/md/md.c > index 194580f..d91920d 100644 > --- a/drivers/md/md.c > +++ b/drivers/md/md.c > @@ -305,7 +305,7 @@ static blk_qc_t md_make_request(struct request_queue *q, > struct bio *bio) > */ > void mddev_suspend(struct mddev *mddev) > { > - WARN_ON_ONCE(current == mddev->thread->tsk); > + WARN_ON_ONCE(mddev->thread && current == mddev->thread->tsk); > if (mddev->suspended++) > return; > synchronize_rcu(); > -- > 2.5.5 > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH] md: raid5: add prerequisite to run underneath dm-raid
From: Heinz MauelshagenIn case md runs underneath the dm-raid target, the mddev does not have a request queue or gendisk, thus avoid accesses. This patch adds a missing conditional to the raid5 personality. Signed-of-by: Heinz Mauelshagen --- drivers/md/raid5.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 8ab8b65..ce79ce6 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -7574,8 +7574,10 @@ static void raid5_finish_reshape(struct mddev *mddev) if (mddev->delta_disks > 0) { md_set_array_sectors(mddev, raid5_size(mddev, 0, 0)); - set_capacity(mddev->gendisk, mddev->array_sectors); - revalidate_disk(mddev->gendisk); + if (mddev->queue) { + set_capacity(mddev->gendisk, mddev->array_sectors); + revalidate_disk(mddev->gendisk); + } } else { int d; spin_lock_irq(>device_lock); -- 2.5.5 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH] md: raid10: add prerequisite to run underneath dm-raid
From: Heinz MauelshagenIn case md runs underneath the dm-raid target, the mddev does not have a request queue or gendisk, thus avoid accesses to it. This patch adds two missing conditionals to the raid10 personality. Signed-of-by: Heinz Mauelshagen --- drivers/md/raid10.c | 12 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/drivers/md/raid10.c b/drivers/md/raid10.c index e3fd725..84e24e6 100644 --- a/drivers/md/raid10.c +++ b/drivers/md/raid10.c @@ -3782,8 +3782,10 @@ static int raid10_resize(struct mddev *mddev, sector_t sectors) return ret; } md_set_array_sectors(mddev, size); - set_capacity(mddev->gendisk, mddev->array_sectors); - revalidate_disk(mddev->gendisk); + if (mddev->queue) { + set_capacity(mddev->gendisk, mddev->array_sectors); + revalidate_disk(mddev->gendisk); + } if (sectors > mddev->dev_sectors && mddev->recovery_cp > oldsize) { mddev->recovery_cp = oldsize; @@ -4593,8 +4595,10 @@ static void raid10_finish_reshape(struct mddev *mddev) set_bit(MD_RECOVERY_NEEDED, >recovery); } mddev->resync_max_sectors = size; - set_capacity(mddev->gendisk, mddev->array_sectors); - revalidate_disk(mddev->gendisk); + if (mddev->queue) { + set_capacity(mddev->gendisk, mddev->array_sectors); + revalidate_disk(mddev->gendisk); + } } else { int d; for (d = conf->geo.raid_disks ; -- 2.5.5 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] [PATCH 4/6] libmultipath: Fixup 'DM_DEVICE_RELOAD' handling
On Mon, May 09, 2016 at 12:26:36PM +0200, Hannes Reinecke wrote: > On 05/06/2016 10:12 PM, Benjamin Marzinski wrote: > > On Wed, May 04, 2016 at 07:57:28AM +0200, Hannes Reinecke wrote: > >> libdevmapper has the 'quirk' that DM_DEVICE_CREATE is translated > >> internally into a create/load/resume sequence, and the associated > >> cookie will wait for the last 'resume' to complete. > >> However, DM_DEVICE_RELOAD has no such translation, so if there > >> is a cookie assigned to it the caller _cannot_ wait for it, > >> as the cookie will only ever be completed upon the next > >> DM_DEVICE_RESUME. > >> multipathd already has some provisions for that (but even there > >> the cookie handling is dodgy), but 'multipath -r' doesn't know > >> about this. > >> So to avoid any future irritations this patch updates the > >> dm_addmad_reload() call to handle the cookie correctly, > >> and removes the special handling from multipathd. > > > > I don't see what's multipathd specific about any of the handling here. > > The real answer is that device-mapper does nothing with cookies on > > table reload. We should never be calling dm_task_set_cookie() for > > dm_addmap(DM_DEVICE_RELOAD, ...) calls in the first place. We end up > > creating a cookie, decrementing the cookie, incrementing the cookie, and > > finally waiting on it, when we could just be creating a cookie and then > > waiting on it. > > > > It's kind of hard to find an easy to show example of this breaking. You > > would need to have the dm_addmap() command fail with some other error than > > EROFS. If that happens, the dm_simplecmd() call will never happen, and > > there will be a cookie sitting around on the system. If we never added > > a cookie to the task in dm_addmap(DM_DEVICE_RELOAD, ...), then there > > wouldn't be this cookie sitting around. > > > But then ... how is this supposed to work? > Are we supposed to call DM_DEVICE_RESUME even after DM_DEVICE_RELOAD > failed? > None of the other callers do that ... No. If we fail to load a new table, we do nothing, and the device continues as it is. If we don't ever call dm_task_set_cookie when we reload the table, then there is no cookie to wait for. And if loading a new table fails, then there is no new table to switch to, and no point in calling resume. -Ben > Cheers, > > Hannes > -- > Dr. Hannes Reinecke Teamlead Storage & Networking > h...@suse.de +49 911 74053 688 > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg > GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton > HRB 21284 (AG Nürnberg) -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH 2/7] devmapper: do not call dm_udev_complete if dm_task_set_cookie() failed
As per Benjamin Marzinski libdevice-mapper will take care of cleaning up the cookie for us. So there's no need to call dm_udev_complete() if dm_task_set_cookie() fails. Signed-off-by: Hannes Reinecke--- libmultipath/devmapper.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index 2ac62a5..8a89f46 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -232,10 +232,9 @@ dm_simplecmd (int task, const char *name, int no_flush, int need_sync, uint16_t #endif if (udev_wait_flag && !dm_task_set_cookie(dmt, , - DM_UDEV_DISABLE_LIBRARY_FALLBACK | udev_flags)) { - dm_udev_complete(cookie); + DM_UDEV_DISABLE_LIBRARY_FALLBACK | udev_flags)) goto out; - } + r = dm_task_run (dmt); if (udev_wait_flag) { @@ -320,10 +319,9 @@ dm_addmap (int task, const char *target, struct multipath *mpp, if (task == DM_DEVICE_CREATE && !dm_task_set_cookie(dmt, , - DM_UDEV_DISABLE_LIBRARY_FALLBACK)) { - dm_udev_complete(cookie); + DM_UDEV_DISABLE_LIBRARY_FALLBACK)) goto freeout; - } + r = dm_task_run (dmt); if (task == DM_DEVICE_CREATE) { -- 2.6.6 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH 4/7] devmapper: wait for udev in dm_simplecmd_noflush()
When calling dm_simplecmd_noflush() with udev_flags set we need to set the 'need_sync' flag otherwise the udev flags will never be set. Signed-off-by: Hannes Reinecke--- libmultipath/configure.c | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/libmultipath/configure.c b/libmultipath/configure.c index b976da7..9c6904a 100644 --- a/libmultipath/configure.c +++ b/libmultipath/configure.c @@ -628,7 +628,7 @@ domap (struct multipath * mpp, char * params) r = dm_addmap_reload(mpp, params); if (r) r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, -0, MPATH_UDEV_RELOAD_FLAG); +1, MPATH_UDEV_RELOAD_FLAG); break; case ACT_RESIZE: @@ -646,7 +646,9 @@ domap (struct multipath * mpp, char * params) if (r) { r = dm_addmap_reload(mpp, params); if (r) - r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, 0, MPATH_UDEV_RELOAD_FLAG); + r = dm_simplecmd_noflush(DM_DEVICE_RESUME, +mpp->alias, 1, + MPATH_UDEV_RELOAD_FLAG); } break; -- 2.6.6 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
[dm-devel] [PATCH 7/7] libmultipath: Fixup 'DM_DEVICE_RELOAD' handling
libdevmapper has the 'quirk' that DM_DEVICE_CREATE is translated internally into a create/load/resume sequence, and the associated cookie will wait for the last 'resume' to complete. However, DM_DEVICE_RELOAD has no such translation, so if there is a cookie assigned to it the caller _cannot_ wait for it, as the cookie will only ever be completed upon the next DM_DEVICE_RESUME. multipathd already has some provisions for that (but even there the cookie handling is dodgy), but 'multipath -r' doesn't know about this. So to avoid any future irritations this patch updates the dm_addmad_reload() call to handle the call to DM_DEVICE_RESUME correctly and removes the special handling from domap(). Signed-off-by: Hannes Reinecke--- libmultipath/configure.c | 18 -- libmultipath/devmapper.c | 31 --- libmultipath/devmapper.h | 2 +- 3 files changed, 29 insertions(+), 22 deletions(-) diff --git a/libmultipath/configure.c b/libmultipath/configure.c index 708dae8..a4a2c44 100644 --- a/libmultipath/configure.c +++ b/libmultipath/configure.c @@ -625,16 +625,11 @@ domap (struct multipath * mpp, char * params) break; case ACT_RELOAD: - r = dm_addmap_reload(mpp, params); - if (r) - r = dm_simplecmd_noflush(DM_DEVICE_RESUME, mpp->alias, -MPATH_UDEV_RELOAD_FLAG); + r = dm_addmap_reload(mpp, params, 0); break; case ACT_RESIZE: - r = dm_addmap_reload(mpp, params); - if (r) - r = dm_simplecmd_flush(DM_DEVICE_RESUME, mpp->alias, 0); + r = dm_addmap_reload(mpp, params, 1); break; case ACT_RENAME: @@ -643,13 +638,8 @@ domap (struct multipath * mpp, char * params) case ACT_FORCERENAME: r = dm_rename(mpp->alias_old, mpp->alias); - if (r) { - r = dm_addmap_reload(mpp, params); - if (r) - r = dm_simplecmd_noflush(DM_DEVICE_RESUME, -mpp->alias, - MPATH_UDEV_RELOAD_FLAG); - } + if (r) + r = dm_addmap_reload(mpp, params, 0); break; default: diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index 6983ab6..6d1a5d6 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -312,7 +312,8 @@ dm_addmap (int task, const char *target, struct multipath *mpp, if (mpp->attribute_flags & (1 << ATTR_GID) && !dm_task_set_gid(dmt, mpp->gid)) goto freeout; - condlog(4, "%s: addmap [0 %llu %s %s]", mpp->alias, mpp->size, + condlog(4, "%s: %s [0 %llu %s %s]", mpp->alias, + task == DM_DEVICE_RELOAD ? "reload" : "addmap", mpp->size, target, params); dm_task_no_open_count(dmt); @@ -371,12 +372,28 @@ dm_addmap_create (struct multipath *mpp, char * params) { #define ADDMAP_RO 1 extern int -dm_addmap_reload (struct multipath *mpp, char *params) { - if (dm_addmap(DM_DEVICE_RELOAD, TGT_MPATH, mpp, params, ADDMAP_RW)) - return 1; - if (errno != EROFS) - return 0; - return dm_addmap(DM_DEVICE_RELOAD, TGT_MPATH, mpp, params, ADDMAP_RO); +dm_addmap_reload (struct multipath *mpp, char *params, int flush) +{ + int r; + uint16_t udev_flags = flush ? 0 : MPATH_UDEV_RELOAD_FLAG; + + /* +* DM_DEVICE_RELOAD cannot wait on a cookie, as +* the cookie will only ever be released after an +* DM_DEVICE_RESUME. So call DM_DEVICE_RESUME +* after each successful call to DM_DEVICE_RELOAD. +*/ + r = dm_addmap(DM_DEVICE_RELOAD, TGT_MPATH, mpp, params, ADDMAP_RW); + if (!r) { + if (errno != EROFS) + return 0; + r = dm_addmap(DM_DEVICE_RELOAD, TGT_MPATH, mpp, + params, ADDMAP_RO); + } + if (r) + r = dm_simplecmd(DM_DEVICE_RESUME, mpp->alias, flush, +1, udev_flags, 0); + return r; } extern int diff --git a/libmultipath/devmapper.h b/libmultipath/devmapper.h index bc13b07..b5df369 100644 --- a/libmultipath/devmapper.h +++ b/libmultipath/devmapper.h @@ -18,7 +18,7 @@ int dm_drv_version (unsigned int * version, char * str); int dm_simplecmd_flush (int, const char *, uint16_t); int dm_simplecmd_noflush (int, const char *, uint16_t); int dm_addmap_create (struct multipath *mpp, char *params); -int dm_addmap_reload (struct multipath *mpp, char *params); +int dm_addmap_reload (struct multipath *mpp, char *params, int flush); int dm_map_present (const char *); int dm_get_map(const char *, unsigned long long *,
[dm-devel] [PATCH 1/7] devmapper: do not flush I/O for DM_DEVICE_CREATE
DM_DEVICE_CREATE loads a new table, so there cannot be any I/O pending. Hence we should be setting the 'no flush' and 'skip lockfs' flag to avoid delays during creation. Signed-off-by: Hannes Reinecke--- libmultipath/devmapper.c | 25 - 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/libmultipath/devmapper.c b/libmultipath/devmapper.c index fb202e8..2ac62a5 100644 --- a/libmultipath/devmapper.c +++ b/libmultipath/devmapper.c @@ -285,16 +285,23 @@ dm_addmap (int task, const char *target, struct multipath *mpp, if (ro) dm_task_set_ro(dmt); - if ((task == DM_DEVICE_CREATE) && strlen(mpp->wwid) > 0){ - prefixed_uuid = MALLOC(UUID_PREFIX_LEN + strlen(mpp->wwid) + 1); - if (!prefixed_uuid) { - condlog(0, "cannot create prefixed uuid : %s", - strerror(errno)); - goto addout; + if (task == DM_DEVICE_CREATE) { + if (strlen(mpp->wwid) > 0) { + prefixed_uuid = MALLOC(UUID_PREFIX_LEN + + strlen(mpp->wwid) + 1); + if (!prefixed_uuid) { + condlog(0, "cannot create prefixed uuid : %s", + strerror(errno)); + goto addout; + } + sprintf(prefixed_uuid, UUID_PREFIX "%s", mpp->wwid); + if (!dm_task_set_uuid(dmt, prefixed_uuid)) + goto freeout; } - sprintf(prefixed_uuid, UUID_PREFIX "%s", mpp->wwid); - if (!dm_task_set_uuid(dmt, prefixed_uuid)) - goto freeout; + dm_task_skip_lockfs(dmt); +#ifdef LIBDM_API_FLUSH + dm_task_no_flush(dmt); +#endif } if (mpp->attribute_flags & (1 << ATTR_MODE) && -- 2.6.6 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel
Re: [dm-devel] [PATCH 4/6] libmultipath: Fixup 'DM_DEVICE_RELOAD' handling
On 05/06/2016 10:12 PM, Benjamin Marzinski wrote: > On Wed, May 04, 2016 at 07:57:28AM +0200, Hannes Reinecke wrote: >> libdevmapper has the 'quirk' that DM_DEVICE_CREATE is translated >> internally into a create/load/resume sequence, and the associated >> cookie will wait for the last 'resume' to complete. >> However, DM_DEVICE_RELOAD has no such translation, so if there >> is a cookie assigned to it the caller _cannot_ wait for it, >> as the cookie will only ever be completed upon the next >> DM_DEVICE_RESUME. >> multipathd already has some provisions for that (but even there >> the cookie handling is dodgy), but 'multipath -r' doesn't know >> about this. >> So to avoid any future irritations this patch updates the >> dm_addmad_reload() call to handle the cookie correctly, >> and removes the special handling from multipathd. > > I don't see what's multipathd specific about any of the handling here. > The real answer is that device-mapper does nothing with cookies on > table reload. We should never be calling dm_task_set_cookie() for > dm_addmap(DM_DEVICE_RELOAD, ...) calls in the first place. We end up > creating a cookie, decrementing the cookie, incrementing the cookie, and > finally waiting on it, when we could just be creating a cookie and then > waiting on it. > > It's kind of hard to find an easy to show example of this breaking. You > would need to have the dm_addmap() command fail with some other error than > EROFS. If that happens, the dm_simplecmd() call will never happen, and > there will be a cookie sitting around on the system. If we never added > a cookie to the task in dm_addmap(DM_DEVICE_RELOAD, ...), then there > wouldn't be this cookie sitting around. > But then ... how is this supposed to work? Are we supposed to call DM_DEVICE_RESUME even after DM_DEVICE_RELOAD failed? None of the other callers do that ... Cheers, Hannes -- Dr. Hannes ReineckeTeamlead Storage & Networking h...@suse.de +49 911 74053 688 SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton HRB 21284 (AG Nürnberg) -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel