[dm-devel] [PATCH] multipath-tools: minor edition and corrections in multipath.conf.5

[Xose Vazquez Perez]

Cc: Christophe Varoqui 
Cc: device-mapper development 
Signed-off-by: Xose Vazquez Perez 
---
 multipath/multipath.conf.5 | 62 +++---
 1 file changed, 36 insertions(+), 26 deletions(-)

diff --git a/multipath/multipath.conf.5 b/multipath/multipath.conf.5
index d9ac279..a78ec5b 100644
--- a/multipath/multipath.conf.5
+++ b/multipath/multipath.conf.5
@@ -2,12 +2,13 @@
 .\" Update the date below if you make any significant change.
 .\" Make sure there are no errors with:
 .\" groff -z -wall -b -e -t multipath/multipath.conf.5
+.\" man --warnings -E UTF-8 -l -Tutf8 -Z multipath/multipath.conf.5 >/dev/null
 .\"
 .\" TODO: Look for XXX and ???
 .\"
 .\" 

 .
-.TH MULTIPATH.CONF 5 2017-05-11 "Linux"
+.TH MULTIPATH.CONF 5 2017-08-18 Linux
 .
 .
 .\" 

@@ -26,6 +27,8 @@ is the configuration file for the multipath daemon. It is 
used to
 overwrite the built-in configuration table of \fBmultipathd\fP.
 Any line whose first non-white-space character is a '#' is considered
 a comment line. Empty lines are ignored.
+.PP
+Internal multipathd configuration can be displayed with the \fBmultipath -t\fR 
command.
 .
 .
 .\" 

@@ -215,11 +218,9 @@ unique identifier in config file, and it would activate 
merging uevents
 according to the identifier to promote effiecncy in processing uevents.
 It has no default value, if you want to identify path by udev attribute
 and to activate merging uevents for SCSI and DASD devices, you can set
-its value as:
+its value as: \fIuid_attrs "sd:ID_SERIAL dasd:ID_UID"\fR.
 .RS
 .TP
-\fBuid_attrs "sd:ID_SERIAL dasd:ID_UID"\fR
-.TP
 The default is: \fB\fR
 .RE
 .
@@ -241,9 +242,13 @@ The default is: for NVME devices \fBID_WWN\fR
 .B getuid_callout
 (Superseded by \fIuid_attribute\fR) The default program and args to callout
 to obtain a unique path identifier. Should be specified with an absolute path.
+.RS
 .TP
+The default is: \fB\fR
+.RE
 .
 .
+.TP
 .B prio
 The name of the path priority routine. The specified routine
 should return a numeric value specifying the relative priority
@@ -301,7 +306,7 @@ Requires prio_args keyword.
 .TP
 .I datacore
 (Hardware-dependent)
-Generate the path priority for some Datacore storage arrays. Requires prio_args
+Generate the path priority for some DataCore storage arrays. Requires prio_args
 keyword.
 .TP
 .I iet
@@ -328,25 +333,24 @@ Needs a value of the form
 .RS
 .TP 8
 .I hbtl
-regex can be of SCSI H:B:T:L format. For example: 1:0:.:. , *:0:0:.
+Regex can be of SCSI H:B:T:L format. For example: 1:0:.:. , *:0:0:.
 .TP
 .I devname
-regex can be of device name format. For example: sda , sd.e
+Regex can be of device name format. For example: sda , sd.e
 .TP
 .I serial
-regex can be of serial number format. For example: .*J1FR.*324 . The serial can
+Regex can be of serial number format. For example: .*J1FR.*324 . The serial can
 be looked up through sysfs or by running multipathd show paths format "%z". For
 example: 0395J1FR904324
 .TP
 .I wwn
-regex can be of the form \fI"host_wwnn:host_wwpn:target_wwnn:target_wwpn"\fR
+Regex can be of the form \fI"host_wwnn:host_wwpn:target_wwnn:target_wwpn"\fR
 these values can be looked up through sysfs or by running \fImultipathd show 
paths format
 "%N:%R:%n:%r"\fR. For example: 0x200100e08ba0aea0:0x210100e08ba0aea0:.*:.* , 
.*:.*:iqn.2009-10.com.redhat.msp.lab.ask-06:.*
 .RE
 .TP 12
 .I path_latency
-Needs a value of the form
-\fI"|"\fR
+Needs a value of the form \fI"|"\fR
 .RS
 .TP 8
 .I io_num
@@ -369,12 +373,21 @@ If \fIexclusive_pref_bit\fR is set, paths with the 
\fIpreferred path\fR bit
 set will always be in their own path group.
 .TP
 .I datacore
-\fIpreferredsds\fR (required) denotes the preferred "SDS name" for datacore
-arrays. \fItimeout\fR (optional) is the timeout for the INQUIRY, in ms.
+.RS
+.TP 8
+.I preferredsds
+(Mandatory) The preferred "SDS name".
 .TP
+.I timeout
+(Optional) The timeout for the INQUIRY, in ms.
+.RE
+.TP 12
 .I iet
-\fIpreferredip=...\fR (required) denotes the preferred IP address (in dotted 
decimal
-notation) for iSCSI targets.
+.RS
+.TP 8
+.I preferredip=...
+(Mandatory) Th preferred IP address, in dotted decimal notation, for iSCSI 
targets.
+.RE
 .TP
 The default is: \fB\fR
 .RE
@@ -387,30 +400,27 @@ where \fInum\fR is the number, between 0 and 8, of 
features in \fIlist\fR.
 Possible values for the feature list are:
 .RS
 .TP 12
-.\" XXX
 .I queue_if_no_path
 (Deprecated, superseded by \fIno_path_retry\fR) Queue I/O if no path is active.
 Identical to the \fIno_path_retry\fR with \fIqueue\fR value. If both this
 feature and \fIno_path_retry\fR are set, the latter value takes
 precedence. See KNOWN ISSUES.
 .TP
-.\" XXX
 .I 

Re: [dm-devel] [PATCH] dm-crypt: limit the number of allocated pages

[John Stoffel]

> "Mikulas" == Mikulas Patocka  writes:

Mikulas> On Mon, 14 Aug 2017, John Stoffel wrote:

>> > "Mikulas" == Mikulas Patocka  writes:
>> 
Mikulas> dm-crypt consumes excessive amount memory when the user attempts to 
zero
Mikulas> a dm-crypt device with "blkdiscard -z". The command "blkdiscard -z" 
calls
Mikulas> the BLKZEROOUT ioctl, it goes to the function __blkdev_issue_zeroout,
Mikulas> __blkdev_issue_zeroout sends large amount of write bios that contain 
the
Mikulas> zero page as their payload.
>> 
Mikulas> For each incoming page, dm-crypt allocates another page that holds the
Mikulas> encrypted data, so when processing "blkdiscard -z", dm-crypt tries to
Mikulas> allocate the amount of memory that is equal to the size of the device.
Mikulas> This can trigger OOM killer or cause system crash.
>> 
Mikulas> This patch fixes the bug by limiting the amount of memory that dm-crypt
Mikulas> allocates to 2% of total system memory.
>> 
>> Is this limit per-device or system wide?  it's not clear to me if the
>> minimum is just one page, or more so it might be nice to clarify
>> that.

Mikulas> The limit is system-wide. The limit is divided by the number
Mikulas> of active dm-crypt devices and each device receives an equal
Mikulas> share.

Ok, thanks for the clarification.  Can it be expanded dynamically?  I
could see that for large systems, it might not scale well.  

Mikulas> There is a lower bound of BIO_MAX_PAGES * 16. The per-device
Mikulas> limit is never lower than this.

Nice.

>> Also, for large memory systems, that's still alot of pages.  Maybe it
>> should be more exponential in it's clamping as memory sizes go up?  2%
>> of 2T is 4G, which is pretty darn big...

Mikulas> The more we restrict it, the less parallelism is used in dm-crypt, so 
it 
Mikulas> makes sense to have a big value on machines with many cores.

True... but it's still a concern that it's hardcoded.

Thanks for your replies.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

[dm-devel] [PATCH] multipath-tools: support for coalescing heterogenous paths

[Guan Junxiong]

This patch supports coalescing heterogenous paths by using
udev rules files. This is useful in the scenario of migrating
data for heterogenous arrays without interrupting uplayer
transaction. The udev rules file, which is located in the
/etc/udev/rules.d/ directory, is automatically generated by
multipath with new options -m and -n.

For example:
multipath -m dm-0 -n  136006016028

The geneated udev rules is as follows:
KERNEL=="sd*[!0-9]|nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk",
ENV{ID_SERIAL}=="136006016028", ENV{ID_SERIAL}="36006016-
028f03600588db521dc7ce71"

KERNEL=="sd*[!0-9]|nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk",
ENV{ID_WWN}=="136006016028", ENV{ID_WWN}="36006016028f036-
00588db521dc7ce71"

KERNEL=="sd*[!0-9]|nvme*[0-9]n*[0-9]", ENV{DEVTYPE}=="disk",
ENV{ID_UUID}=="136006016028", ENV{ID_UUID}="36006016028f03-
600588db521dc7ce71"

Signed-off-by: Junxiong Guan 
---
 libmultipath/Makefile   |   3 +-
 libmultipath/config.h   |   1 +
 libmultipath/udevrule.c | 191 
 libmultipath/udevrule.h |  13 
 multipath/main.c|  29 +++-
 multipath/multipath.8   |  12 +++
 6 files changed, 247 insertions(+), 2 deletions(-)
 create mode 100644 libmultipath/udevrule.c
 create mode 100644 libmultipath/udevrule.h

diff --git a/libmultipath/Makefile b/libmultipath/Makefile
index b3244fc7..ae2fdc2d 100644
--- a/libmultipath/Makefile
+++ b/libmultipath/Makefile
@@ -42,7 +42,8 @@ OBJS = memory.o parser.o vector.o devmapper.o callout.o \
pgpolicies.o debug.o defaults.o uevent.o time-util.o \
switchgroup.o uxsock.o print.o alias.o log_pthread.o \
log.o configure.o structs_vec.o sysfs.o prio.o checkers.o \
-   lock.o waiter.o file.o wwids.o prioritizers/alua_rtpg.o
+   lock.o waiter.o file.o wwids.o prioritizers/alua_rtpg.o \
+   udevrule.o
 
 all: $(LIBS)
 
diff --git a/libmultipath/config.h b/libmultipath/config.h
index ffc69b5f..1f8438b7 100644
--- a/libmultipath/config.h
+++ b/libmultipath/config.h
@@ -34,6 +34,7 @@ enum mpath_cmds {
CMD_REMOVE_WWID,
CMD_RESET_WWIDS,
CMD_ADD_WWID,
+   CMD_GEN_UDEVRULE,
 };
 
 enum force_reload_types {
diff --git a/libmultipath/udevrule.c b/libmultipath/udevrule.c
new file mode 100644
index ..e927bf9a
--- /dev/null
+++ b/libmultipath/udevrule.c
@@ -0,0 +1,191 @@
+/*
+ * udevrule.c
+ *
+ * Copyright (c) 2017 Guan Junxiong , Huawei
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * See the file COPYING included with this distribution for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "vector.h"
+#include "structs.h"
+#include "debug.h"
+#include "uxsock.h"
+#include "file.h"
+#include "udevrule.h"
+
+#define UUID_PREFIX "mpath-"
+#define UUID_PREFIX_LEN 6
+
+static int has_wwid(FILE *f, char *wwid)
+{
+   char buf[LINE_MAX];
+
+   while (fgets(buf, LINE_MAX, f) != NULL) {
+   if (strstr(buf, wwid))
+   return 1;
+   }
+
+   return 0;
+}
+
+static ssize_t get_md_uuid(char *md, char *dm_uuid)
+{
+   int fd;
+   ssize_t size = -1;
+   char devpath[PATH_SIZE];
+
+   if (strlen(md) < 4 || strncmp(md, "dm-", 3)) {
+   condlog(0, "%s: device name should start with \"dm-\"", md);
+   return -EINVAL;
+   }
+   snprintf(devpath, PATH_SIZE, "/sys/devices/virtual/block/%s/dm/uuid",
+   md);
+   fd = open(devpath, O_RDONLY);
+   if (fd < 0) {
+   condlog(4, "attribute '%s' can not be opened: %s",
+   devpath, strerror(errno));
+   return -errno;
+   }
+   size = read(fd, dm_uuid, WWID_SIZE);
+   if (size < 0) {
+   condlog(4, "read from %s failed: %s", devpath, strerror(errno));
+   size = -errno;
+   dm_uuid[0] = '\0';
+   } else {
+   dm_uuid[size] = '\0';
+   }
+
+   if (!strncmp(dm_uuid, UUID_PREFIX, UUID_PREFIX_LEN)) {
+   memmove(dm_uuid, dm_uuid + UUID_PREFIX_LEN,
+   strlen(dm_uuid + UUID_PREFIX_LEN) + 1);
+   size -= UUID_PREFIX_LEN;
+   }
+   close(fd);
+
+   return size;
+}
+
+static int write_udev_rule(int fd, char *kname, char *dm_wwid,
+   char *wwid, char *id_type)
+{
+   int ret;
+   off_t offset;
+   char buf[LINE_MAX];
+
+