Re: [dm-devel] [PATCH v3 29/29] libmultipath: fix race between log_safe and log_thread_stop()

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:17:08PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > log_safe() could race with log_thread_stop(); simply > checking the value of log_thr has never been safe. By converting the > mutexes to static initializers, we avoid having to destroy them, and thus > possi

[dm-devel] multipath-tools build and unit testing

2020-12-16 Thread Martin Wilck
Hello all, I've set up a collection of Dockerfiles under https://github.com/mwilck/build-multipath. They can be used to build multipath-tools quickly for various distributions and run the unit tests, which is handy for avoiding regressions. Feedback and contributions welcome, cheers, Martin --

Re: [dm-devel] [PATCH v2 5/6] libmultipath: limit reading 0xc9 vpd page

2020-12-16 Thread Martin Wilck
On Wed, 2020-12-16 at 17:56 -0600, Benjamin Marzinski wrote: > On Wed, Dec 16, 2020 at 09:18:01PM +, Martin Wilck wrote: > > > > Do we still need the name check after testing whether 0xc9 is > > supported? Well I guess it doesn't harm. > > I understand that people could want to use the device

Re: [dm-devel] [PATCH v3 25/29] libmultipath: introduce symbolic values for logsink

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:17:04PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > Reviewed-by: Benjamin Marzinski > Signed-off-by: Martin Wilck > --- > libmultipath/debug.c | 4 ++-- > libmultipath/debug.h | 6 ++ > libmultipath/devmapper.c | 4 ++-- > multipath/main.c

Re: [dm-devel] [PATCH v3 24/29] libmultipath: use libmp_verbosity to track verbosity

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:17:03PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > Introduce a new global variable to set the verbosity of libmultipath. > This avoids accessing the configuration in every dlog() call. > When libmultipath reads its configuration in init_config() or > load_co

Re: [dm-devel] [PATCH v3 22/29] multipath: fix leaks in check_path_valid()

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:17:01PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > There were two leaks in check_path_valid(): if path status was > successfully determined before calling store_pathvec(), free_path() > wasn't called. Also, if an error exit occured, neither cleanup > functio

Re: [dm-devel] [PATCH v3 20/29] multipath: use atexit() for cleanup handlers

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:16:59PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > Signed-off-by: Martin Wilck Reviewed-by: Benjamin Marzinski > --- > multipath/main.c | 37 - > 1 file changed, 16 insertions(+), 21 deletions(-) > > diff --git a/multi

Re: [dm-devel] [PATCH resend] multipath-tools: Violin and Nexsan were bought by StorCentric

2020-12-16 Thread Martin Wilck
On Wed, 2020-12-16 at 23:17 +0100, Xose Vazquez Perez wrote: > Reviewed-by: Martin Wilck > Cc: Martin Wilck > Cc: Benjamin Marzinski > Cc: Christophe Varoqui > Cc: DM-DEVEL ML > Signed-off-by: Xose Vazquez Perez > --- > libmultipath/hwtable.c | 7 +++ > 1 file changed, 3 insertions(+), 4

Re: [dm-devel] [PATCH v3 18/29] libmultipath: fix log_thread startup and teardown

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:16:57PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > This fixes several issues with the log_thread. First, the running flag > logq_running should be set by the thread itself, not by > log_thread_start()/_stop(). Second, the thread was both cancelled and > term

Re: [dm-devel] [PATCH v3 02/29] multipathd: Fix liburcu memory leak

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 07:16:41PM +0100, mwi...@suse.com wrote: > From: Martin Wilck > > Fix this leak in multipathd, reported by valgrind, that messes up > multipathd's otherwise clean leak report: > > ==23823== 336 bytes in 1 blocks are possibly lost in loss record 3 of 3 > ==23823==at 0x

Re: [dm-devel] [PATCH v2 5/6] libmultipath: limit reading 0xc9 vpd page

2020-12-16 Thread Martin Wilck
On Wed, 2020-11-04 at 00:54 -0600, Benjamin Marzinski wrote: > Only rdac arrays support 0xC9 vpd page inquiries. All other arrays > will > return a failure. Only do the rdac inquiry when detecting array > capabilities if the array's path checker is explicitly set to rdac, > or > the path checker is

Re: [dm-devel] [PATCH v2 4/6] libmultipath: factor out code to get vpd page data

2020-12-16 Thread Martin Wilck
On Wed, 2020-11-04 at 00:54 -0600, Benjamin Marzinski wrote: > A future patch will reuse the code to get the vpd page data, so > factor > it out from get_vpd_sgio(). > > Signed-off-by: Benjamin Marzinski > --- > libmultipath/discovery.c | 19 +++ > 1 file changed, 15 insertions(+

Re: [dm-devel] [PATCH v2 2/6] libmultipath: add eh_deadline multipath.conf parameter

2020-12-16 Thread Martin Wilck
On Wed, 2020-11-04 at 00:54 -0600, Benjamin Marzinski wrote: > There are times a fc rport is never lost, meaning that > fast_io_fail_tmo > and dev_loss_tmo never trigger, but scsi commands still hang. This > can > cause problems in cases where users have strict timing requirements, > and > the easi

Re: [dm-devel] [PATCH v3 0/4] add library to check if device is a valid path

2020-12-16 Thread Martin Wilck
Hi Ben: On Wed, 2020-10-21 at 16:39 -0500, Benjamin Marzinski wrote: > The main part of the this patchset is the first patch, which adds a > new library interface to check whether devices are valid paths. This > was designed for use in the Storage Instantiation Daemon (SID). > > https://github.co

Re: [dm-devel] [PATCH v2 5/6] libmultipath: limit reading 0xc9 vpd page

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 09:18:01PM +, Martin Wilck wrote: > On Wed, 2020-11-04 at 00:54 -0600, Benjamin Marzinski wrote: > > Only rdac arrays support 0xC9 vpd page inquiries. All other arrays > > will > > return a failure. Only do the rdac inquiry when detecting array > > capabilities if the ar

Re: [dm-devel] [PATCH v2 4/6] libmultipath: factor out code to get vpd page data

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 09:13:49PM +, Martin Wilck wrote: > On Wed, 2020-11-04 at 00:54 -0600, Benjamin Marzinski wrote: > > A future patch will reuse the code to get the vpd page data, so > > factor > > it out from get_vpd_sgio(). > > > > Signed-off-by: Benjamin Marzinski > > --- > > libmul

Re: [dm-devel] [PATCH v2 22/29] multipath: fix leaks in check_path_valid()

2020-12-16 Thread Benjamin Marzinski
On Wed, Dec 16, 2020 at 06:34:05PM +0100, Martin Wilck wrote: > On Fri, 2020-10-16 at 12:44 +0200, mwi...@suse.com wrote: > > From: Martin Wilck > > > > There were two leaks in check_path_valid(): if path status was > > successfully determined before calling store_pathvec(), free_path() > > wasn'

Re: [dm-devel] [PATCH v3 00/29] libmultipath: improve cleanup on exit

2020-12-16 Thread Martin Wilck
On Wed, 2020-12-16 at 19:16 +0100, mwi...@suse.com wrote: > > This series is based on the previous series "multipath-tools: > shutdown, > libdevmapper races, globals" (v3). > I forgot to mention: The previous 2 series "multipath-tools: shutdown, libdevmapper races, globals" (v3) "mult

[dm-devel] [PATCH v3 25/29] libmultipath: introduce symbolic values for logsink

2020-12-16 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- libmultipath/debug.c | 4 ++-- libmultipath/debug.h | 6 ++ libmultipath/devmapper.c | 4 ++-- multipath/main.c | 4 ++-- multipathd/main.c| 17 - tests/globals.c | 3 ++- tests/hwtable.

[dm-devel] [PATCH v3 24/29] libmultipath: use libmp_verbosity to track verbosity

2020-12-16 Thread mwilck
From: Martin Wilck Introduce a new global variable to set the verbosity of libmultipath. This avoids accessing the configuration in every dlog() call. When libmultipath reads its configuration in init_config() or load_config(), it will use the current value of libmp_verbosity for logging. Immedia

[dm-devel] [PATCH v3 27/29] multipathd: common code for "-k" and command args

2020-12-16 Thread mwilck
From: Martin Wilck 'multipathd -k"cmd"' and 'multipath cmd' are the same thing. Treat it with common code. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 37 +++-- 1 file changed, 19 insertions(+), 18 deletions(-) diff --gi

[dm-devel] [PATCH v3 22/29] multipath: fix leaks in check_path_valid()

2020-12-16 Thread mwilck
From: Martin Wilck There were two leaks in check_path_valid(): if path status was successfully determined before calling store_pathvec(), free_path() wasn't called. Also, if an error exit occured, neither cleanup function was called. This patch fixes both, at the cost of using "static" for the p

[dm-devel] [PATCH v3 29/29] libmultipath: fix race between log_safe and log_thread_stop()

2020-12-16 Thread mwilck
From: Martin Wilck log_safe() could race with log_thread_stop(); simply checking the value of log_thr has never been safe. By converting the mutexes to static initializers, we avoid having to destroy them, and thus possibly accessing a destroyed mutex in log_safe(). Furthermore, taking both the l

[dm-devel] [PATCH v3 12/29] multipathd: unwatch_all_dmevents: check if waiter is initialized

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/dmevents.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/multipathd/dmevents.c b/multipathd/dmevents.c index b561cbf..f52f597 100644 --- a/multipathd/dmevents.c +++ b/multipathd/dmevents.c @@

[dm-devel] [PATCH v3 18/29] libmultipath: fix log_thread startup and teardown

2020-12-16 Thread mwilck
From: Martin Wilck This fixes several issues with the log_thread. First, the running flag logq_running should be set by the thread itself, not by log_thread_start()/_stop(). Second, the thread was both cancelled and terminated via a flag (again, logq_running). It's sufficient to just cancel and j

[dm-devel] [PATCH v3 26/29] libmultipath: simplify dlog()

2020-12-16 Thread mwilck
From: Martin Wilck By checking the log level in condlog() directly, we can simplify dlog(). Also, it's now possible to limit the log level at compile time by setting MAX_VERBOSITY, enabling the compiler to optimize away log messages with higher loglevel. Reviewed-by: Benjamin Marzinski Signed-o

[dm-devel] [PATCH v3 23/29] multipath-tools: mpath-tools.supp: file with valgrind suppressions

2020-12-16 Thread mwilck
From: Martin Wilck These leaks are caused by other libraries (libsystemd, glibc, libgcrypt) and should be ignored when debugging with valgrind Usage example: valgrind --suppressions=mpath-tools.supp \ --leak-check=full --show-leak-kinds=all $COMMAND Reviewed-by: Benjamin Marzinski Signed-

[dm-devel] [PATCH v3 28/29] multipathd: sanitize uxsock_listen()

2020-12-16 Thread mwilck
From: Martin Wilck We were allocating 1025 poll fds, which is weird. Change it to a power of two, and make this more easily customizable in general. Use POLLFDS_BASE rather than the hard-coded "2" for the number of fds we poll besides client connections. Introduce a maximum number of clients tha

[dm-devel] [PATCH v3 03/29] multipathd: move handling of io_err_stat_attr into libmultipath

2020-12-16 Thread mwilck
From: Martin Wilck This thread attribute can be dynamically initialized and destroyed. No need to carry it along in multipathd. Removal of the symbol requires to bump the ABI version to 3. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmultipath/io_err_stat.c| 7 +

[dm-devel] [PATCH v3 14/29] libmultipath: add libmp_dm_exit()

2020-12-16 Thread mwilck
From: Martin Wilck This function prepares for calling dm_lib_exit() on program exit. It undoes changes to libdm internals done by libmultipath. It doesn't call dm_lib_exit(), as the caller may want to keep libdm active. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmultip

[dm-devel] [PATCH v3 21/29] mpathpersist: use atexit() for cleanup handlers

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- mpathpersist/main.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/mpathpersist/main.c b/mpathpersist/main.c index 3c2e657..14245cc 100644 --- a/mpathpersist/main.c +++ b/mpathpersist/m

[dm-devel] [PATCH v3 13/29] multipathd: print error message if config can't be loaded

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index 6b9e323..7ab3eab 100644 --- a/multipathd/main.c +++ b/multipathd/main.c @@ -30

[dm-devel] [PATCH v3 16/29] libmultipath: log_thread_stop(): check if logarea is initialized

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmultipath/log_pthread.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/libmultipath/log_pthread.c b/libmultipath/log_pthread.c index 15baef8..0c327ff 100644 --- a/libmultipath/log_pthread.c +++ b/libm

[dm-devel] [PATCH v3 01/29] multipathd: uxlsnr: avoid deadlock on exit

2020-12-16 Thread mwilck
From: Martin Wilck The uxlsnr wouldn't always release the client lock when cancelled, causing a deadlock in uxsock_cleanup(). While this hasn't been caused by commit 3d611a2, the deadlock seems to have become much more likely after that patch. Solving this means that we have to treat reallocation

[dm-devel] [PATCH v3 19/29] multipathd: move cleanup_{prio, checkers, foreign} to libmultipath_exit

2020-12-16 Thread mwilck
From: Martin Wilck This requires another major ABI bump. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- libmpathpersist/mpath_persist.c | 2 -- libmultipath/config.c | 4 libmultipath/libmultipath.version | 5 + multipathd/main.c | 3 ---

[dm-devel] [PATCH v3 05/29] multipathd: make some globals static

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 3da0d7c..eb760a7 100644 --- a/multipathd/main.c +++ b/multipathd/m

[dm-devel] [PATCH v3 17/29] multipathd: add cleanup_child() exit handler

2020-12-16 Thread mwilck
From: Martin Wilck cleanup_child() calls all cleanups in the right order, in an exit handler. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 45 + 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/m

[dm-devel] [PATCH v3 07/29] multipathd: move conf destruction into separate function

2020-12-16 Thread mwilck
From: Martin Wilck Also removing the comment about dlog() and dm_write_log(). dlog() can cope with get_multipath_config() returning NULL, and dm_write_log() hasn't accessed the configuration for a while. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 20 ++

[dm-devel] [PATCH v3 04/29] multipathd: move vecs desctruction into cleanup function

2020-12-16 Thread mwilck
From: Martin Wilck This will make it easer to move the stuff around later. The only functional change is that map destuction now happens after joining all threads, which should actually improve robustness. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 64

[dm-devel] [PATCH v3 20/29] multipath: use atexit() for cleanup handlers

2020-12-16 Thread mwilck
From: Martin Wilck Signed-off-by: Martin Wilck --- multipath/main.c | 37 - 1 file changed, 16 insertions(+), 21 deletions(-) diff --git a/multipath/main.c b/multipath/main.c index 9ae46ed..1949a1c 100644 --- a/multipath/main.c +++ b/multipath/main.c @@ -452

[dm-devel] [PATCH v3 11/29] multipathd: child(): call cleanups in failure case, too

2020-12-16 Thread mwilck
From: Martin Wilck So far we haven't called any cleanup code if child() failed. Fix it. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 10 -- 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index

[dm-devel] [PATCH v3 06/29] multipathd: move threads destruction into separate function

2020-12-16 Thread mwilck
From: Martin Wilck Also, introduce booleans that indicate a certain thread has been started successfully. Using these booleans, we can avoid crashing by cancelling threads that have never been started. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 76

[dm-devel] [PATCH v3 15/29] multipathd: fixup libdm deinitialization

2020-12-16 Thread mwilck
From: Martin Wilck With libmp_dm_exit() in place, we can make sure that the calls are made in the right order. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/multipathd/main.c b/multipa

[dm-devel] [PATCH v3 02/29] multipathd: Fix liburcu memory leak

2020-12-16 Thread mwilck
From: Martin Wilck Fix this leak in multipathd, reported by valgrind, that messes up multipathd's otherwise clean leak report: ==23823== 336 bytes in 1 blocks are possibly lost in loss record 3 of 3 ==23823==at 0x483AB65: calloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so) ==2

[dm-devel] [PATCH v3 08/29] multipathd: move pid destruction into separate function

2020-12-16 Thread mwilck
From: Martin Wilck Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 10 +++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/multipathd/main.c b/multipathd/main.c index 07973e8..fc1f8d7 100644 --- a/multipathd/main.c +++ b/multipathd/main.

[dm-devel] [PATCH v3 09/29] multipathd: close pidfile on exit

2020-12-16 Thread mwilck
From: Martin Wilck It seems we've been doing this only in the failure case, for ages. Reviewed-by: Benjamin Marzinski Signed-off-by: Martin Wilck --- multipathd/main.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/multipathd/main.c b/multipathd/main.c index fc1f8d7..f

[dm-devel] [PATCH v3 00/29] libmultipath: improve cleanup on exit

2020-12-16 Thread mwilck
From: Martin Wilck Hi Christophe, hi Ben, hi lixiaokeng, this series was inspired by lixiaokeng's recent posting "[QUESTION] memory leak in main (multipath)". It implements my first idea, registering cleanup handlers with atexit(). However it turned out to be quite complex. In particular multipa

[dm-devel] [PATCH resend] multipath-tools: add Vexata(by StorCentric) VX arrays

2020-12-16 Thread Xose Vazquez Perez
https://support.sas.com/resources/papers/performance-tuning-sas-vexata-systems.pdf Reviewed-by: Martin Wilck Cc: Martin Wilck Cc: Benjamin Marzinski Cc: Christophe Varoqui Cc: DM-DEVEL ML Signed-off-by: Xose Vazquez Perez --- libmultipath/hwtable.c | 8 1 file changed, 8 insertions

[dm-devel] [PATCH resend] multipath-tools: Violin and Nexsan were bought by StorCentric

2020-12-16 Thread Xose Vazquez Perez
Reviewed-by: Martin Wilck Cc: Martin Wilck Cc: Benjamin Marzinski Cc: Christophe Varoqui Cc: DM-DEVEL ML Signed-off-by: Xose Vazquez Perez --- libmultipath/hwtable.c | 7 +++ 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/libmultipath/hwtable.c b/libmultipath/hwtable.c ind

Re: [dm-devel] [PATCH v2 22/29] multipath: fix leaks in check_path_valid()

2020-12-16 Thread Martin Wilck
On Fri, 2020-10-16 at 12:44 +0200, mwi...@suse.com wrote: > From: Martin Wilck > > There were two leaks in check_path_valid(): if path status was > successfully determined before calling store_pathvec(), free_path() > wasn't called. Also, if an error exit occured, neither cleanup > function was c

[dm-devel] [PATCH] dm-raid: set discard_granularity non-zero if possible

2020-12-16 Thread Stephan Bärwolf
Hi I hope this address is the right place for this patch. It is supposed to fix the triggering of block/blklib.c:51 WARN_ON_ONCE(..) when using LVM2 raid1 with SSD-PVs. Since commit b35fd7422c2f8e04496f5a770bd4e1a205414b3f and without this patchthere are tons of printks logging "Error: discard_

Re: [dm-devel] [PATCH v1 0/5] dm: dm-user: New target that proxies BIOs to userspace

2020-12-16 Thread Vitaly Mayatskih
On Mon, Dec 14, 2020 at 10:03 PM Palmer Dabbelt wrote: > I was really experting someone to say that. It does seem kind of silly to > build > out the new interface, but not go all the way to a ring buffer. We just > didn't > really have any way to justify the extra complexity as our use cases