Bug#852712: iscsistart -N errors with be2iscsi driver

2017-01-26 Thread Andrew Patterson
Package: open-iscsi
Version: 2.0.874-2

The following error messages are issued when running iscsistart -N
with a device using the be2iscsi driver:

 # iscsistart -N
 iscsistart: Could not get driver .
 Setting up software interface
 iscsistart: No netdev name in fw entry.
 Setting up software interface failed

A subsequent run of iscsistart -f and iscsistart -b works fine. The
iscsstart -N routines should probable bypass be2iscsi devices given
that Emulex driver/firmware sets up networking on it own.

--
Andrew Patterson Hewlett-Packard Enterprise



Bug#850060: Boot from iSCSI patches

2017-01-24 Thread Andrew Patterson
Here is another set of patches to get boot from iSCSI working on
debian with bnx2x devices. They are:

* Revert "Don't ignore offloading NICs in iscsistart."
* Check for the presence of /sbin/iscsiuio before using hardware
  offload for bnx2x devices.
* Move iscsistart offload discovery/setup to fw_get_targets()
* Fix broken long command-line options in iscsiuio.

I have tested these patches using bnx2x devices configured as both
software initiators and using hardware offload.

-- 
Andrew Patterson
Hewlett-Packard Enterprise
>From 17b79c2a25a9ba5a105661f49bf435148ff30277 Mon Sep 17 00:00:00 2001
From: Andrew Patterson 
Date: Tue, 17 Jan 2017 10:25:37 -0700
Subject: [PATCH 4/7] Revert "Don't ignore offloading NICs in iscsistart."

This reverts commit 02c3051a7718dc26c71bd682da1f8d44358ffa96.
---
 .../iscsistart-offloading-interfaces.patch | 24 --
 debian/patches/series  |  1 -
 2 files changed, 25 deletions(-)
 delete mode 100644 debian/patches/bugfixes/iscsistart-offloading-interfaces.patch

diff --git a/debian/patches/bugfixes/iscsistart-offloading-interfaces.patch b/debian/patches/bugfixes/iscsistart-offloading-interfaces.patch
deleted file mode 100644
index d9a383c..000
--- a/debian/patches/bugfixes/iscsistart-offloading-interfaces.patch
+++ /dev/null
@@ -1,24 +0,0 @@
-Description: Don't ignore offloading NICs in iscsistart
-Author: Christian Seiler 
-Bug: https://groups.google.com/forum/?_escaped_fragment_=msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ#!msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ
-Bug-Debian: https://bugs.debian.org/850057
-Last-Update: 2017-01-04

-This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
 a/utils/fwparam_ibft/fw_entry.c
-+++ b/utils/fwparam_ibft/fw_entry.c
-@@ -65,10 +65,14 @@ int fw_setup_nics(void)
- 	 * to force iSCSI traffic through correct NIC
- 	 */
- 	list_for_each_entry(context, &targets, list) {			
-+		/* Debian: don't ignore offload NICs here, see Debian
-+		 * bug #850057 for details. */
-+#if 0
- 		/* if it is a offload nic ignore it */
- 		if (!net_get_transport_name_from_netdev(context->iface,
- 			transport))
- 			continue;
-+#endif
- 
- 		if (iface_prev == NULL || strcmp(context->iface, iface_prev)) {
- 			/* Note: test above works because there is a
diff --git a/debian/patches/series b/debian/patches/series
index a109715..6a2a532 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -2,4 +2,3 @@ bugfixes/no-make-clean-kernel.patch
 debian/var-run-lock.patch
 debian/dont-link-against-openssl.patch
 debian/udeb-without-libmount.patch
-bugfixes/iscsistart-offloading-interfaces.patch
-- 
2.8.0.rc3

>From 1f6a5beb286a03fbba82da053f648f9c52c721c5 Mon Sep 17 00:00:00 2001
From: Andrew Patterson 
Date: Thu, 19 Jan 2017 15:26:48 -0700
Subject: [PATCH 5/7] iscsiuio must be present to use hardware offload for
 bnx2x

Check for the presence of /sbin/iscsiuio before using hardware offload
for bnx2x devices.
---
 .../need_iscsiuio_for_hardware_offload.patch   | 48 ++
 debian/patches/series  |  1 +
 2 files changed, 49 insertions(+)
 create mode 100644 debian/patches/bugfixes/need_iscsiuio_for_hardware_offload.patch

diff --git a/debian/patches/bugfixes/need_iscsiuio_for_hardware_offload.patch b/debian/patches/bugfixes/need_iscsiuio_for_hardware_offload.patch
new file mode 100644
index 000..0bb0d57
--- /dev/null
+++ b/debian/patches/bugfixes/need_iscsiuio_for_hardware_offload.patch
@@ -0,0 +1,48 @@
+Description: iscsiuio must be present to use hardware offload for bnx2x
+Author: Andrew Patterson 
+Bug: https://groups.google.com/forum/?_escaped_fragment_=msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ#!msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ
+Bug-Debian: https://bugs.debian.org/850057
+Last-Update: 2017-01-24
+---
+This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
+--- a/include/iscsi_net_util.h
 b/include/iscsi_net_util.h
+@@ -2,6 +2,7 @@
+ #define __ISCSI_NET_UTIL_h__
+ 
+ #define ISCSI_HWADDRESS_BUF_SIZE 18
++#define ISCSIUIO_PATH "/sbin/iscsiuio"
+ 
+ extern int net_get_transport_name_from_netdev(char *netdev, char *transport);
+ extern int net_get_netdev_from_hwaddress(char *hwaddress, char *netdev);
+--- a/usr/iscsi_net_util.c
 b/usr/iscsi_net_util.c
+@@ -27,6 +27,7 @@
+ #include 
+ #include 
+ #include 
++#include 
+ #include 
+ #include 
+ #include 
+@@ -81,6 +82,20 @@
+ 		goto close_sock;
+ 	}
+ 
++	/*
++	 * iSCSI hardware offload for bnx2x is only supported if the
++	 * iscsiuio executable is available.
++	 */
++	if (!strcmp(drvinfo.driver, "bnx2x")) {
++		struct stat buf;
++
++		if (stat(ISCSIUIO_PATH, &buf)) {
++			log_debug(1, "ISCSI offload not supported.");
++			err = ENODEV;
++			goto close_sock;
++		}
++	}
++
+ 	for (i = 0; net_drivers[i].net_drv_name != NULL; i++) {
+ 		struct iscsi_net_driver *net_dri

Bug#850060: 2.0.874-2~exp1 issues

2017-01-05 Thread Andrew Patterson

On 01/05/2017 12:10 PM, Christian Seiler wrote:
> On 01/05/2017 07:10 PM, Andrew Patterson wrote:
>> On 01/04/2017 04:11 PM, Christian Seiler wrote:
>>> OTOH, initramfs should write to /run/initramfs only, so maybe
>>> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio
>>> instead as well.
>>
>> Yes. I believe that will work. I wonder why this option is not in the
>> man-page.
> 
> -ENOTIME on part of the developers perhaps. ;-)
> 
>>> The ideal solution would be to mirror the check that is done
>>> for -b in -N. In that case we'd either configure the host
>>> interface (and use software iSCSI), or configure offloading
>>> (and use hardware iSCSI), but never both, and never neither.
>>> So instead of the current patch for #850057 I would suggest
>>> to do that instead. That should then also be upstream-able. I
>>> can prepare a patch for that tomorrow.

Sounds good.

>>
>> That was my thought. However, I don't think you can programaticaly
>> determine whether the card is configured for iscsi offload.
> 
> Well, according to the commit message I referenced in my
> previous email [1], we have the following situation:
> 
>  - cxgb*i: always use offloading (which will apparently work,
>since offloading will reuse the MAC address)
> 
>  - bnx2*i: look at the MAC address to see if the MAC address
>from iBFT matches the offloading mac: if so, assume
>offloading, if not, assume software
> 
>  - otherwise: always assume software
> 
> This is what the code bracketed by -DOFFLOAD_BOOT_SUPPORTED
> checks for, and we just need to mirror that code (or better:
> extract into an own function and call it from both places)
> for -b.
> 
> I'll prepare a patch, you can then test it.
> 
> [1] 
> https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff
> 
>> It looks like iscsuio is complaining about a missing libgcc:
>>
>> writev(2, [{"libgcc_s.so.1 must be installed "..., 59]], 1) = 59
>>
>> which is indeed missing.  I think pthreads needs libgcc, but shouldn't
>> copy_exec take care of this?
> 
> Nope, since libgcc_s.so is dlopened() by pthread_cancel. So
> the automatic library dependency detection doesn't work
> properly.
> 
> We are not the only ones with that problem; luckily someone
> already wrote a patch that does just this for btrfs, we can
> just steal that:
> https://bugs.debian.org/830883

I tested this "fix". iscsiuio/iscsistart -b is working now.

> 
> I'll work on an updated package.
> 
> Regards,
> Christian
> 

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850060: 2.0.874-2~exp1 issues

2017-01-05 Thread Andrew Patterson

On 01/05/2017 11:10 AM, Andrew Patterson wrote:
> 
> On 01/04/2017 04:11 PM, Christian Seiler wrote:
>> Control: reopen -1
>>
>> Hi,
>> (reopening the bug report since iscsiuio doesn't actually work
>> according to what you're telling me)
>>
>> On 01/04/2017 11:30 PM, Andrew Patterson wrote:
>>> 1. In debian/extra/initramfs.local-top and
>>> debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file
>>> in /run/iscsiuio.pid. You cannot override it on the command-line. The
>>> --pidfile option for start-stop-daemon should point to this location
>>> instead of /run/initramfs/iscsiuio.pid.
>>
>> Gah. I thought I had fixed that before the upload. :-(
>> A good thing there's experimental...
>>
>> OTOH, initramfs should write to /run/initramfs only, so maybe
>> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio
>> instead as well.
> 
> Yes. I believe that will work. I wonder why this option is not in the
> man-page.
> 
>>
>>> 2. The /sbin/iscsiuio deamon is trying to create a logfile in
>>> /var/log/iscsiuio.log. The /var/log directory does not exist in the
>>> initramfs.
>>
>> Unfortunately that's hardcoded. OTOH from reading the code
>> this shouldn't be an issue.
>>
>>> 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am
>>> investigating why. It works fine when run after boot.
>>
>> This is weird, on my test VM (no offloading NIC though) it
>> does start (and shut down again).
>>
>> Question: does the network card require firmware for offloading
>> to work? If that's the case we might need to add the firmware
>> to the initramfs or this to work properly...
> 
> Yes, the card requires firmware. But it has been loaded (according to
> kernel output).
> 
>>
>>> 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart"
>>> will cause the interface to be configured in both iscsistart -N and in
>>> iscsistart -b for cards with iscsi hardware offload. I suspect we
>>> instead need a flag in iscsistart to bring up the NIC in iscsistart
>>> regardless if the NIC uses an offloadable driver. The local-top script
>>> can use this flag if /sbin/iscsiuio is not present, e.g.,
>>>
>>> ISCSISTART_FLAGS=""
>>> if [ ! -x /sbin/iscsiuio ] ; then
>>>ISCSISTART_FLAGS="-S"
>>> fi
>>
>> Hmm, now I see what you mean. I also dug up this commit which
>> gives a little insight.
>>
>> https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff
>>
>> However, I think the solution is different from what you suggest: the
>> "fix" for #850057 is wrong I believe.
>>
>> Debian sid package (w/o -DOFFLOAD_BOOT_SUPPORTED):
>>
>>  - cxgb*i: iscsistart -N ignores interface
>>iscsistart -b doesn't configure offloading
>>=> won't work
>>
>>  - bnx2i (any mode): iscsistart -N ignores interface
>>  iscsistart -b doesn't configure offloading
>>=> won't work
>>
>> Current upstream situation (w/ -DOFFLOAD_BOOT_SUPPORTED):
>>
>>  - cxgb*i: iscsistart -N ignores interface
>>iscsistart -b configures offloading 
>>=> should work
>>   (but don't have hardware)
>>   (also: remind me to check whether we copy the
>>   module into the initramfs)
>>
>>  - bnx2i non-hba: iscsistart -N ignores it
>>   iscsistart -b doesn't configure offloading
>>=> won't work
>>
>>  - bnx2i hba: iscsistart -N ignores it
>>   iscsistart -b configures offloading
>>=> should work (once iscsiuio is running)
>>
>> "Fix" for 850057 (in experimental):
>>
>>  - cxgb*i: iscsistart -N configures interface on host
>>iscsistart -b configures offloading
>>  => no idea what happens thereafter
>> (since MAC address is the same on these cards: is
>> this a problem if they have the same IP?)
>>
>>  - bnx2i non-hba: iscsistart -N configures interface
>>   iscsistart -b doesn't configure offloading,
>> but enables software iSCSI
>>   => should work
>>
>>  - bnx2i hba: iscsistart -N configures interface
>>   iscsistart -b configures offloading
>>   

Bug#850060: 2.0.874-2~exp1 issues

2017-01-05 Thread Andrew Patterson

On 01/04/2017 04:11 PM, Christian Seiler wrote:
> Control: reopen -1
> 
> Hi,
> (reopening the bug report since iscsiuio doesn't actually work
> according to what you're telling me)
> 
> On 01/04/2017 11:30 PM, Andrew Patterson wrote:
>> 1. In debian/extra/initramfs.local-top and
>> debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file
>> in /run/iscsiuio.pid. You cannot override it on the command-line. The
>> --pidfile option for start-stop-daemon should point to this location
>> instead of /run/initramfs/iscsiuio.pid.
> 
> Gah. I thought I had fixed that before the upload. :-(
> A good thing there's experimental...
> 
> OTOH, initramfs should write to /run/initramfs only, so maybe
> we should pass -p /run/initramfs/iscsiuio.pid to iscsiuio
> instead as well.

Yes. I believe that will work. I wonder why this option is not in the
man-page.

> 
>> 2. The /sbin/iscsiuio deamon is trying to create a logfile in
>> /var/log/iscsiuio.log. The /var/log directory does not exist in the
>> initramfs.
> 
> Unfortunately that's hardcoded. OTOH from reading the code
> this shouldn't be an issue.
> 
>> 3. The iscsiuio daemon is dieing before or during iscsistart -b. I am
>> investigating why. It works fine when run after boot.
> 
> This is weird, on my test VM (no offloading NIC though) it
> does start (and shut down again).
> 
> Question: does the network card require firmware for offloading
> to work? If that's the case we might need to add the firmware
> to the initramfs or this to work properly...

Yes, the card requires firmware. But it has been loaded (according to
kernel output).

> 
>> 4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart"
>> will cause the interface to be configured in both iscsistart -N and in
>> iscsistart -b for cards with iscsi hardware offload. I suspect we
>> instead need a flag in iscsistart to bring up the NIC in iscsistart
>> regardless if the NIC uses an offloadable driver. The local-top script
>> can use this flag if /sbin/iscsiuio is not present, e.g.,
>>
>> ISCSISTART_FLAGS=""
>> if [ ! -x /sbin/iscsiuio ] ; then
>>ISCSISTART_FLAGS="-S"
>> fi
> 
> Hmm, now I see what you mean. I also dug up this commit which
> gives a little insight.
> 
> https://github.com/open-iscsi/open-iscsi/commit/ee115be828362653478e6fe7cd4c6ee3318223ff
> 
> However, I think the solution is different from what you suggest: the
> "fix" for #850057 is wrong I believe.
> 
> Debian sid package (w/o -DOFFLOAD_BOOT_SUPPORTED):
> 
>  - cxgb*i: iscsistart -N ignores interface
>iscsistart -b doesn't configure offloading
>=> won't work
> 
>  - bnx2i (any mode): iscsistart -N ignores interface
>  iscsistart -b doesn't configure offloading
>=> won't work
> 
> Current upstream situation (w/ -DOFFLOAD_BOOT_SUPPORTED):
> 
>  - cxgb*i: iscsistart -N ignores interface
>iscsistart -b configures offloading 
>=> should work
>   (but don't have hardware)
>   (also: remind me to check whether we copy the
>   module into the initramfs)
> 
>  - bnx2i non-hba: iscsistart -N ignores it
>   iscsistart -b doesn't configure offloading
>=> won't work
> 
>  - bnx2i hba: iscsistart -N ignores it
>   iscsistart -b configures offloading
>=> should work (once iscsiuio is running)
> 
> "Fix" for 850057 (in experimental):
> 
>  - cxgb*i: iscsistart -N configures interface on host
>iscsistart -b configures offloading
>  => no idea what happens thereafter
> (since MAC address is the same on these cards: is
> this a problem if they have the same IP?)
> 
>  - bnx2i non-hba: iscsistart -N configures interface
>   iscsistart -b doesn't configure offloading,
> but enables software iSCSI
>   => should work
> 
>  - bnx2i hba: iscsistart -N configures interface
>   iscsistart -b configures offloading
>   => same IP, different MAC addresses
>   => will cause trouble
> 
> The ideal solution would be to mirror the check that is done
> for -b in -N. In that case we'd either configure the host
> interface (and use software iSCSI), or configure offloading
> (and use hardware iSCSI), but never both, and never neither.
> So instead of the current patch for #850057 I would suggest
> to do that instead. That should then also be upstrea

Bug#850060: 2.0.874-2~exp1 issues

2017-01-04 Thread Andrew Patterson
I have been testing 2.0.874-2~exp1 and found the following issues:

1. In debian/extra/initramfs.local-top and
debian/extra/initramfs.local-bottom /sbin/iscsuio creates a pid file
in /run/iscsiuio.pid. You cannot override it on the command-line. The
--pidfile option for start-stop-daemon should point to this location
instead of /run/initramfs/iscsiuio.pid.

2. The /sbin/iscsiuio deamon is trying to create a logfile in
/var/log/iscsiuio.log. The /var/log directory does not exist in the
initramfs. I do not know if this is causing a problem (perhaps related
to #1).

3. The iscsiuio daemon is dieing before or during iscsistart -b. I am
investigating why. It works fine when run after boot.

4. The commit 02c3051 "Don't ignore offloading NICs in iscsistart"
will cause the interface to be configured in both iscsistart -N and in
iscsistart -b for cards with iscsi hardware offload. I suspect we
instead need a flag in iscsistart to bring up the NIC in iscsistart
regardless if the NIC uses an offloadable driver. The local-top script
can use this flag if /sbin/iscsiuio is not present, e.g.,

ISCSISTART_FLAGS=""
if [ ! -x /sbin/iscsiuio ] ; then
   ISCSISTART_FLAGS="-S"
fi
.
.
.
modprobe iscsi_ibft
iscsistart -N $ISCSISTART_FLAGS

The iscsistart code would have to be modified to add support for this
flag. Alternatively, iscsistart could check for the presence of
/sbin/iscsiuio and "do the right thing".

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850060: open-iscsi initramfs scripts do not start up iscsiuio deamon

2017-01-04 Thread Andrew Patterson
On 01/03/2017 03:47 PM, Christian Seiler wrote:
> On 01/03/2017 10:51 PM, Andrew Patterson wrote:
>> On 01/03/2017 01:27 PM, Christian Seiler wrote:
>>> On 01/03/2017 09:10 PM, Andrew Patterson wrote:
>>>>> How does the rest of the boot process proceed then? What happens
>>>>> when iscsiuio is to be started regularly at boot from the systemd
>>>>> service / init script?
>>>>> Is the iscsiuio from the initramfs required
>>>>> to be running at all times during the iSCSI session or can it be
>>>>> restarted, as long as the time during which that happens is brief
>>>>> enough? Is it required to be kept around during shutdown if it's
>>>>> on iSCSI?
>>>>
>>>> I don't believe it needs to be running to support traffic, just when
>>>> doing logins. I will test to confirm.
>>>
>>> That would be ideal. :)
>>
>> I tried running a load with a data integrity check while starting and
>> stopping iscsiuio numerous times. I did not see any issues either
>> with performance or data integrity.
> 
> Did you try stopping iscsiuio for longer periods of time? For
> example more than 20 minutes? I would like to know what happens
> if you hit ARP timeouts.

I ran it overnight (13 hours) with no issues.

> 
>>> As far as I can tell from skimming the source iscsiuio is also
>>> responsible for doing ARP and responding to ICMP  - but even
>>> then it wouldn't be a problem for restarting in that case, as
>>> long as the MAC address of the target (or a router on the way
>>> to the target) doesn't change during restart.
>>>
>>
>> It this likely to be an issue in the one or two seconds between
>> transitioning between the initramfs and sysvinit/systemd?
> 
> It's only a couple of seconds if there's no fsck running for
> another partition. Then it could easily be minutes or even
> longer, depending on the partition size, the number of files
> and the filesystem in question.
>

Granted.

>>>>> Does it follow systemd's process name convention to make
>>>>> sure systemd doesn't kill it during shutdown?
>>
>> I don't see any special handling of iscsiuio in systemd shutdown.
> 
> I meant this:
> 
> https://www.freedesktop.org/wiki/Software/systemd/RootStorageDaemons/
> (The @-stuff is only relevant if the binary is _not_ started
> via a systemd service.)

This only applies for systemd though. It won't help us for sysv-init
(assuming we need to care).

> 
> OTOH, since iscsiuio is apparently not required to be running
> while data is transferred, killing it at shutdown deosn't
> seem to be that big of a deal.

It seems so.

> 
>> It looks like we can use start-stop-daemon in local-top and
>> local-bottom to gracefully start and stop the daemon.
> 
> Yes, that appears to be the case.
> 
>>>>> Before I add the aforementioned steps 1 & 2 I'd like to be sure that
>>>>> this actually works properly beyond the initramfs. So any
>>>>> information related to that would be appreciated.
>>>>
>>
>> Is my current testing enough or do we need to do more? If the later,
>> what sort of test would you like me to run?
> 
> See my request above: if after 20 minutes of iscsiuio being
> offline everything is still fine, I'd be fine with it. If it's
> less, it depends on how much less.

It looks like we are good. At least in the normal case.

> 
> That said: your initial tests are sufficient for me to upload
> a version of open-iscsi that includes support for this to
> the 'experimental' suite, so I'll do that either tomorrow or
> on Thursday, depending on when I'll find time for that. If you
> could then test this (and write down how you set this up so
> we can add it to README.Debian), and give me feedback on the
> "iscsiuio is offline for longer" test, then I'd be fine with
> uploading this to unstable so it makes it into Stretch.
>

I would be happy to test it.

> Other than #850057: does iscsistart need additional changes to
> work with iscsiuio? One would probably need to pass
> -P iface.transport_name=bnx2i -P iface.net_ifacename=eth0
> -P iface.hwaddress=aa:bb:cc:dd:ee:ff or similar, right?
>

I have been using iscsstart -b with iSCSI IBFT to bring up the
interface which seems to do all the required magic (at least for the
iscsi_auto case). If not using iscsi_auto, the local_top script wants iSCSI
login parameters on the kernel command-line.

Don't forget to define OFFLOAD_BOOT_SUPPORTED, ala:

diff --git a/debian/rules b/debian/rules
index 906132a..562ef8d 100755
--- a/debian/rules
+++ b/debian/rules
@@ -3,7 +3,7 @@
 #export DH_VERBOSE=1
 
 export DEB_BUILD_MAINT_OPTIONS = hardening=+all
-export DEB_CFLAGS_MAINT_APPEND = -Wall
+export DEB_CFLAGS_MAINT_APPEND = -Wall -DOFFLOAD_BOOT_SUPPORTED
 
 DPKG_EXPORT_BUILDFLAGS = 1
 include /usr/share/dpkg/default.mk


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850060: open-iscsi initramfs scripts do not start up iscsiuio deamon

2017-01-03 Thread Andrew Patterson
On 01/03/2017 01:27 PM, Christian Seiler wrote:
> On 01/03/2017 09:10 PM, Andrew Patterson wrote:
>>> How does the rest of the boot process proceed then? What happens
>>> when iscsiuio is to be started regularly at boot from the systemd
>>> service / init script?
>>> Is the iscsiuio from the initramfs required
>>> to be running at all times during the iSCSI session or can it be
>>> restarted, as long as the time during which that happens is brief
>>> enough? Is it required to be kept around during shutdown if it's
>>> on iSCSI?
>>
>> I don't believe it needs to be running to support traffic, just when
>> doing logins. I will test to confirm.
> 
> That would be ideal. :)

I tried running a load with a data integrity check while starting and
stopping iscsiuio numerous times. I did not see any issues either
with performance or data integrity.

> 
> As far as I can tell from skimming the source iscsiuio is also
> responsible for doing ARP and responding to ICMP  - but even
> then it wouldn't be a problem for restarting in that case, as
> long as the MAC address of the target (or a router on the way
> to the target) doesn't change during restart.
>

It this likely to be an issue in the one or two seconds between
transitioning between the initramfs and sysvinit/systemd?

>>> Does it follow systemd's process name convention to make
>>> sure systemd doesn't kill it during shutdown?
>>>

I don't see any special handling of iscsiuio in systemd shutdown. Here is the
service file:

[Unit]
Description=iSCSI userspace offloading daemon (iscsiuio)
Documentation=man:iscsiuio(8)
Documentation=file:///usr/share/doc/iscsiuio/README.gz
Wants=network.target
Before=iscsid.service
After=network.target
DefaultDependencies=no
Conflicts=shutdown.target
Before=shutdown.target

[Service]
Type=forking
PIDFile=/run/iscsiuio.pid
ExecStart=/sbin/iscsiuio

[Install]
WantedBy=sysinit.target

>>
>> I am looking into this. One thought is to start the daemon, do the
>> login, and then kill it. Then the normal sysvinit/upstart/systemd
>> scripts can restart it normally.
> 
> If it's really just for logins: yes, we can do that. If it also
> does ARP and such, I'd rather only kill it as close to the restart
> as possible. But having the init script / systemd service (there
> was never a native upstart job for it) kill the initramfs one
> right before starting the own binary is easy, so if that's required
> then that would also be fine.

It looks like we can use start-stop-daemon in local-top and
local-bottom to gracefully start and stop the daemon.

> 
> It's going to be a lot trickier if we can't restart it at all.
> 
>>> Before I add the aforementioned steps 1 & 2 I'd like to be sure that
>>> this actually works properly beyond the initramfs. So any
>>> information related to that would be appreciated.
>>

Is my current testing enough or do we need to do more? If the later,
what sort of test would you like me to run?

>> I am going to look at other package code to see how they handle daemon
>> startup. NFS comes to mind.
> 
> NFS doesn't actually need a daemon running after the initial NFS
> login, at least temporarily:
>

NFS for the initramfs is only used in client mode, so this was a red
herring.

>  - NFSv3 needs the to have portmap running on the client side, so
>that locking works. However, you can spoof that, which is what
>klibc does:
> 
>
> http://sources.debian.net/src/klibc/2.0.4-9/usr/kinit/nfsmount/README.locking/
> 
>Basically you don't need the portmapper to be running at all
>until and actual file lock is taken.
> 
>  - NFSv4 doesn't need anything running on the client side to mount
>with sec=sys with raw UIDs. For idmapping you can use the
>request-key / nfsidmap kernel upcalls instead of idmapd (which
>is deprecated on the client side), so nothing needs to be
>running there either.
> 
>  - NFSv4 + Kerberos needs rpc.gssd running, but I've never seen
>root on Kerberized NFSv4 even being advertised. (You'd also
>need to have credentials with which you can get a Kerberos
>ticket in the initramfs + the tools to get the ticket for
>that to work.)
> 
> Regards,
> Christian
> 

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850060: open-iscsi initramfs scripts do not start up iscsiuio deamon

2017-01-03 Thread Andrew Patterson
On 01/03/2017 01:02 PM, Christian Seiler wrote:
> Control: tags -1 + confirmed moreinfo
> Control: severity -1 wishlist
> 
> On 01/03/2017 06:02 PM, Andrew Patterson wrote:
>> Version 2.0.874 of the open-iscsi package requires that the iscsiuio
>> daemon to be running before iscsi hardware offload cards can login to
>> a LUN. The initramfs scripts used by open-iscsi to support boot from
>> iSCSI do no start up this daemon resulting in a failed boot.
> 
> I'd really like to support this setup. My idea how this could work
> would be along the lines of the following:
> 
>  1. In the local-top hook start iscsiuio before calling iscsistart -b
> or iscsistart $params if the binary is present in the initramfs.
> 
>  2. In the iscsiuio package install a initramfs hook that copies the
> iscsiuio binary (luckily it doesn't depend on any library) into
> the initramfs.
> 
> Together that should start iscsiuio in the initramfs and allow
> login to the targets there - but if iscsiuio is not installed, the
> user won't see a difference from today.
>

That was my plan.

> However, there are still some questions I'd like to have answered
> before proceeding - and since I don't have the corresponding hardware
> I have no idea what the right answers are:
> 
> How does the rest of the boot process proceed then? What happens
> when iscsiuio is to be started regularly at boot from the systemd
> service / init script?
Is the iscsiuio from the initramfs required
> to be running at all times during the iSCSI session or can it be
> restarted, as long as the time during which that happens is brief
> enough? Is it required to be kept around during shutdown if it's
> on iSCSI?

I don't believe it needs to be running to support traffic, just when
doing logins. I will test to confirm.

Does it follow systemd's process name convention to make
> sure systemd doesn't kill it during shutdown?
>

I am looking into this. One thought is to start the daemon, do the
login, and then kill it. Then the normal sysvinit/upstart/systemd
scripts can restart it normally.

> Before I add the aforementioned steps 1 & 2 I'd like to be sure that
> this actually works properly beyond the initramfs. So any
> information related to that would be appreciated.

I am going to look at other package code to see how they handle daemon
startup. NFS comes to mind.


> 
> Regards,
> Christian
> 

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850060: open-iscsi initramfs scripts do not start up iscsiuio deamon

2017-01-03 Thread Andrew Patterson
Package: open-iscsi
Version: 2.0.874-1


Version 2.0.874 of the open-iscsi package requires that the iscsiuio
daemon to be running before iscsi hardware offload cards can login to
a LUN. The initramfs scripts used by open-iscsi to support boot from
iSCSI do no start up this daemon resulting in a failed boot.

Some discussion for this issue can be found at:

https://groups.google.com/forum/?_escaped_fragment_=topic/open-iscsi/PsT65Z4Gx3I#!topic/open-iscsi/PsT65Z4Gx3I

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#850057: iscsistart -N skips offload NICs causing boot to fail

2017-01-03 Thread Andrew Patterson
Package: open-iscsi
Version: 2.0.874-1

The debian initramfs uses the IBFT support in iscsistart to log
into the root volume. The initramfs script uses iscsistart -N to
bring up the NICs before logging in with iscsstart -b. This
process works fine when using non-offload NICs, but fails when
using NICs using the bnx2, bnx2x, cxgb3, and cxgb4 drivers due to
the following code in
utils/fwparam_ibft/fw_entry.c:fw_setup_nics():

list_for_each_entry(context, &targets, list) {
/* if it is a offload nic ignore it */
if (!net_get_transport_name_from_netdev(context->iface,
transport))
continue


Which does a lookup in the table in usr/iscsi_net_util.c

static struct iscsi_net_driver net_drivers[] = {
{"cxgb3", "cxgb3i" },
{"cxgb4", "cxgb4i" },
{"bnx2", "bnx2i" },
{"bnx2x", "bnx2i"},
{NULL, NULL}
};


to see if -N should skip this NIC.

There is a discussion of this issue at:

https://groups.google.com/forum/?_escaped_fragment_=msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ#!msg/open-iscsi/PsT65Z4Gx3I/GUObdlVvCQAJ

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#830159: apt-get autoremove removes kernels with "+" in the package name

2016-07-06 Thread Andrew Patterson
On 07/06/2016 02:38 PM, Andrew Patterson wrote:
> 
> 
> On 07/06/2016 02:04 PM, Julian Andres Klode wrote:
>> On Wed, Jul 06, 2016 at 01:48:17PM -0600, Andrew Patterson wrote:
>>> Package: apt
>>> Version: 1.2.11
>>>
>>> When I run apt-get autoremove, apt will also remove kernels with a "+"
>>> in the package name, e.g,
>> [...]
>>> I have attached a patch that has a fix for this problem.
>>
>> Thanks for your patch!
>>
>> Can you also extend the test in 
>> test/integration/test-kernel-helper-autoremove? That
>> would be helpful to ensure it did not work before and works now. We could 
>> also do it
>> ourselves, but maybe you want to do it (looks more coherent that way!).
>>
> 
> I'll give it a try.
> 
>> Would be great if you could submit an updated patch with that in it (and a 
>> "Closes: #830159" as
>> a last line of the commit message, so we don't forget to close the bug) and 
>> in the right
>> format (see below).
>>
> 
> Will do.
> 
>>>
>>> -- 
>>> Andrew Patterson
>>> Hewlett-Packard Enterprise
>>
>>> commit 4c851c0d9b8d04e6c6aa44d1d380e139f0c2f7af
>>> Author: Andrew Patterson 
>>> Date:   Wed Jul 6 13:40:16 2016 -0600
>>
>> Please format patches with git-format patch, this cannot be applied
>> directly with git am.
>>
> 
> Sorry. I'll be better next time.
> 
> 

Here is the new version, formatted with git format-patch and with the requested 
test integration changes.

BTW, that test framework it pretty slick.

Andrew


-- 
Andrew Patterson
Hewlett-Packard Enterprise
>From 75d3da5fa3c9a19ed7bd90ecd64c95d715f120f8 Mon Sep 17 00:00:00 2001
From: Andrew Patterson 
Date: Wed, 6 Jul 2016 13:40:16 -0600
Subject: [PATCH] Add kernels with "+" in the package name to
 APT::NeverAutoRemove

Escape "+" in kernel package names when generating APT::NeverAutoRemove
list so it is not treated as a regular expression meta-character.

Closes: #830159
---
 debian/apt.auto-removal.sh | 2 +-
 test/integration/test-kernel-helper-autoremove | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/debian/apt.auto-removal.sh b/debian/apt.auto-removal.sh
index e7e2ca0..608d950 100644
--- a/debian/apt.auto-removal.sh
+++ b/debian/apt.auto-removal.sh
@@ -38,7 +38,7 @@ $running_version
 $previous_version" | sort -u | sed -e '/^$/ d')"
 kernels="$( (echo "$1
 $unamer"; for deb in $debkernels; do echo "$list" | awk "\$2 == \"$deb\" { print \$1; }"; done; ) \
-   | sed -e 's#\.#\\.#g' -e '/^$/ d' | sort -u)"
+   | sed -e 's#\([\.\+]\)#\\\1#g' -e '/^$/ d' | sort -u)"
 
 generateconfig() {
 	cat <

Bug#830159: apt-get autoremove removes kernels with "+" in the package name

2016-07-06 Thread Andrew Patterson


On 07/06/2016 02:04 PM, Julian Andres Klode wrote:
> On Wed, Jul 06, 2016 at 01:48:17PM -0600, Andrew Patterson wrote:
>> Package: apt
>> Version: 1.2.11
>>
>> When I run apt-get autoremove, apt will also remove kernels with a "+"
>> in the package name, e.g,
> [...]
>> I have attached a patch that has a fix for this problem.
> 
> Thanks for your patch!
> 
> Can you also extend the test in 
> test/integration/test-kernel-helper-autoremove? That
> would be helpful to ensure it did not work before and works now. We could 
> also do it
> ourselves, but maybe you want to do it (looks more coherent that way!).
> 

I'll give it a try.

> Would be great if you could submit an updated patch with that in it (and a 
> "Closes: #830159" as
> a last line of the commit message, so we don't forget to close the bug) and 
> in the right
> format (see below).
> 

Will do.

>>
>> -- 
>> Andrew Patterson
>> Hewlett-Packard Enterprise
> 
>> commit 4c851c0d9b8d04e6c6aa44d1d380e139f0c2f7af
>> Author: Andrew Patterson 
>> Date:   Wed Jul 6 13:40:16 2016 -0600
> 
> Please format patches with git-format patch, this cannot be applied
> directly with git am.
> 

Sorry. I'll be better next time.


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#830159: apt-get autoremove removes kernels with "+" in the package name

2016-07-06 Thread Andrew Patterson
Package: apt
Version: 1.2.11

When I run apt-get autoremove, apt will also remove kernels with a "+"
in the package name, e.g,

# apt-get -s autoremove
Reading package lists... Done
Building dependency tree   
Reading state information... Done
The following packages will be REMOVED:
  linux-image-3.14.69-2+hlinux1-amd64-hlinux
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
Remv linux-image-3.14.69-2+hlinux1-amd64-hlinux [3.14.69-2+hlinux1]

This is due to the "+" character not being escaped in
/etc/apt/apt.conf.d//etc/apt/apt.conf.d/01autoremove-kernels:

# head /etc/apt/apt.conf.d/01autoremove-kernels 
// DO NOT EDIT! File autogenerated by apt.auto-removal.sh
APT::NeverAutoRemove
{
   "^linux-image-3\.14\.69-2+hlinux1-amd64-hlinux$";
   "^linux-image-4\.4\.11-2-amd64-hpelinux$";
   "^linux-image-4\.4-amd64-hpelinux$";
   "^linux-headers-3\.14\.69-2+hlinux1-amd64-hlinux$";
   "^linux-headers-4\.4\.11-2-amd64-hpelinux$";
   "^linux-headers-4\.4-amd64-hpelinux$";
   "^linux-image-extra-3\.14\.69-2+hlinux1-amd64-hlinux$"; 

The "+" character is being treated as a regexp meta-character instead
of as a literal.  These kernels are obviously from a variant Debian
distribution, but others may run into the same issue.

The 01autoremove-kernels is generated by
/etc/kernel/postinst.d/apt-auto-removal (source file
debian/apt.conf.autoremove). The following code from that file
generates the blacklist:

kernels="$( (echo "$1
$unamer"; for deb in $debkernels; do echo "$list" | awk "\$2 == \"$deb\" { 
print \$1; }"; done; ) \
   | sed -e 's#\.#\\.#g' -e '/^$/ d' | sort -u)"

The sed line is escaping "." characters, but not "+" characters (legal
in package names).

I have attached a patch that has a fix for this problem.

-- 
Andrew Patterson
Hewlett-Packard Enterprise
commit 4c851c0d9b8d04e6c6aa44d1d380e139f0c2f7af
Author: Andrew Patterson 
Date:   Wed Jul 6 13:40:16 2016 -0600

Add kernels with "+" in the package name to APT::NeverAutoRemove

Escape "+" in kernel package names when generating APT::NeverAutoRemove
list so it is not treated as a regular expression meta-character.

diff --git a/debian/apt.auto-removal.sh b/debian/apt.auto-removal.sh
index e7e2ca0..608d950 100644
--- a/debian/apt.auto-removal.sh
+++ b/debian/apt.auto-removal.sh
@@ -38,7 +38,7 @@ $running_version
 $previous_version" | sort -u | sed -e '/^$/ d')"
 kernels="$( (echo "$1
 $unamer"; for deb in $debkernels; do echo "$list" | awk "\$2 == \"$deb\" { print \$1; }"; done; ) \
-   | sed -e 's#\.#\\.#g' -e '/^$/ d' | sort -u)"
+   | sed -e 's#\([\.\+]\)#\\\1#g' -e '/^$/ d' | sort -u)"
 
 generateconfig() {
 	cat <

Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-25 Thread Andrew Patterson
On 06/25/2016 02:16 AM, Ritesh Raj Sarraf wrote:
> On Thu, 2016-06-23 at 16:50 -0600, Andrew Patterson wrote: >> +# Save boot 
> setting for aio-max-nr. >> +BOOT_AIO_MAX_NR=$(cat
/proc/sys/fs/aio-max-nr) >> + >>  if [ -f /etc/multipath-tools-boot.conf
]; then >>  . /etc/multipath-tools-boot.conf >>  else >> 
MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath" >> +   
MULTIPATH_TOOLS_MIN_AIO_MAX_NR=$BOOT_AIO_MAX_NR >>  fi >>  > > I'd let
the user define the AIO_MAX value explicitly in /etc/default/multipath-
> tools. And use the sysctl/procfs option as a fallback. > > What do you
say ? > >

That was where I went initially, but I thought that means that the
user now needs to configure aio-max-nr in two places, once in
sysctl.conf and once in /etc/default/multipath-tools (for when running
multipath tools by hand or in scripts). I thought this solution
resulted in less overall system configuration.  The
/etc/default/multipath-tools is more explicit, so it does have
advantages too.

Andrew

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827407: multipath-tools: initramfs should load same mp modules as sysvinit/sysmtemd files

2016-06-25 Thread Andrew Patterson
On 06/25/2016 02:15 AM, Ritesh Raj Sarraf wrote:
> On Wed, 2016-06-22 at 16:08 -0600, Andrew Patterson wrote: >> if [ -f 
> /etc/multipath-tools-boot.conf ]; then >> +.
/etc/multipath-tools-boot.conf >> +else >> +   
MULTIPATH_TOOLS_HW_HANDLERS="" >> +fi > > Are there good reasons to have
a new config file? > Wouldn't /etc/default/multipath-tools suffice ? >

Do you mean, just copy /etc/default/multipath-tools to the initramfs?
I suppose that would work. They are used somewhat differently in later
patches, but it could be repurposed.


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-23 Thread Andrew Patterson
On Mon, 20 Jun 2016 16:40:29 -0600 Andrew Patterson  
wrote:
> On Thu, 16 Jun 2016 15:19:22 +0530 Ritesh Raj Sarraf  wrote:
> > Control: tag -1 +pending
> > 
> > On Wed, 2016-06-15 at 13:03 -0600, Andrew Patterson wrote:
> > > For most users this is not an issue. How many systems are going to use
> > > that many LUNs?
> > > 
> > 
> > I agree.
> > 
> > > One counter-example is using one or more FC LUNs per VM guest on a
> > > host. In such a case, one can just modify /etc/sysctl.conf to increase
> > > the setting. 
> > 
> > Yes. Most hypervisors may want this. But again, like you said, most users 
> > do not
> > map that many LUNs. And recommended practice is to have a VM backed by a 
> > file
> > backend mostly.
> 
> 
> OpenStack/cinder can (and is) be uses to provision VMs with networked
> drives. It is pretty common for cloud applications.  I don't know what
> a practical limit for the number of volume per VMs per host should be.
> 
> 
> > 
> > > We can run into problems when using
> > > multipath-tools-boot. In this case, we use the default kernel settings
> > > for aio-max-nr since multipath is run from the initramfs. I have
> > > worked around this by modifying the script in
> > > /usr/share/initramfs/scripts/local-top/multipath to temporarilly set
> > > aio-max-nr to 1048576 while doing discovery.
> > 
> > 
> > So, I'll just add this to README.Debian. There's not much that we can do 
> > beyond
> > documenting such behavior.
> 
> I will work on patch to have a user-configurable setting of aio-max-nr
> in the initramfs. You can then decide if it is worth it to apply it.
> 
> In the meantime, the README.Debian change sounds fine to me.
> 

Here is a patch that copies the fs.aio-max-nr setting to the
initramfs. There is more to do to solve the problem with booting with
many LUNs, but this is a prerequisite for completely solving that
issue.  This patch assumes that the patch in:

https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=827407;filename=multipath-tools-827407.patch;msg=30

has been applied. I can rework it if that one is not accepted.

Andrew

-- 
Andrew Patterson
Hewlett-Packard Enterprise
commit e68651ca8b6a9c2b8afee787b3e5221b402846fd
Author: Andrew Patterson 
Date:   Thu Jun 23 22:34:16 2016 +

Use fs.aio-max-nr setting in initramfs

We sometimes need to increase the fs.aio-max-nr from the kernel default
value when running multipath discovery in the initramfs with large numbers
of LUNs. This patch copies the setting there and temporarily changes it
while performing discovery.

diff --git a/debian/initramfs/hooks b/debian/initramfs/hooks
index 94cb7e8..a29dbac 100644
--- a/debian/initramfs/hooks
+++ b/debian/initramfs/hooks
@@ -63,4 +63,4 @@ for x in $MULTIPATH_TOOLS_MODULES; do
 done
 echo "MULTIPATH_TOOLS_HW_HANDLERS=\"$MULTIPATH_TOOLS_HW_HANDLERS\"" > $DESTDIR/etc/multipath-tools-boot.conf
 echo "MULTIPATH_TOOLS_MODULES=\"$MULTIPATH_TOOLS_MODULES\"" >> $DESTDIR/etc/multipath-tools-boot.conf
-
+echo "MULTIPATH_TOOLS_MIN_AIO_MAX_NR=$(sysctl fs.aio-max-nr | cut -f3 -d ' ')" >> $DESTDIR/etc/multipath-tools-boot.conf
diff --git a/debian/initramfs/local-top b/debian/initramfs/local-top
index 99e4222..67698bd 100644
--- a/debian/initramfs/local-top
+++ b/debian/initramfs/local-top
@@ -31,10 +31,14 @@ if [ ! -e /sbin/multipath ]; then
 	exit 0
 fi
 
+# Save boot setting for aio-max-nr.
+BOOT_AIO_MAX_NR=$(cat /proc/sys/fs/aio-max-nr)
+
 if [ -f /etc/multipath-tools-boot.conf ]; then
 . /etc/multipath-tools-boot.conf
 else
 MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+MULTIPATH_TOOLS_MIN_AIO_MAX_NR=$BOOT_AIO_MAX_NR
 fi
 
 verbose && log_begin_msg "Loading multipath modules"
@@ -47,6 +51,12 @@ for module in ${MULTIPATH_TOOLS_MODULES}; do
 done
 verbose && log_end_msg
 
+
+# Adjust aio-max-nr if too low.
+if [ $BOOT_AIO_MAX_NR -le $MULTIPATH_TOOLS_MIN_AIO_MAX_NR ]; then
+echo "$MULTIPATH_TOOLS_MIN_AIO_MAX_NR" > /proc/sys/fs/aio-max-nr
+fi
+
 verbose && log_begin_msg "Discovering multipaths"
 /sbin/multipath -v $VERBOSITY
 verbose && log_end_msg
@@ -59,6 +69,9 @@ if [ -x /sbin/udevadm ]; then
 /sbin/udevadm settle --timeout=10
 fi
 
+# Restore aio-max-nr.
+echo "$BOOT_AIO_MAX_NR" > /proc/sys/fs/aio-max-nr
+
 maybe_break post-multipath
 
 exit 0
diff --git a/debian/multipath-tools.README.Debian b/debian/multipath-tools.README.Debian
index c1f50a2..2665046 100644
--- a/debian/multipath-tools.README.Debian
+++ b/debian/multipath-tools.README.Debian
@@ -35,7 +35,8 @@ To make this setting persistent a line may be added to /etc/sysctl.conf:
 Consult appropriate application and operating system tuning recommendations for
 guidance on appropriate values for this parameter.
 
-
+If using multipath-tools-boot, remember to run update-initramfs so the
+adjusted aio-max-nr will also be set in the initramfs.
 
 
 What are these weird numbers in /dev/mapper?


Bug#827407: multipath-tools: initramfs should load same mp modules as sysvinit/sysmtemd files

2016-06-22 Thread Andrew Patterson
On Wed, 22 Jun 2016 11:18:41 -0600 Andrew Patterson
 wrote:

I forgot to add the multipath-tools.default file in the patch. Here is
the revised patch.

-- 
Andrew Patterson
Hewlett-Packard Enterprise

commit 566cc94c2fbcf1c59e5d3f8c30fd08c567f8a382
Author: Andrew Patterson 
Date:   Wed Jun 22 10:46:50 2016 -0600

Load common set of multipath-modules in initramfs, sysvinit, and systemd

Use variables in /etc/default/multipath-tools to load a common set
of multipath kernel modules and hardware handlers in the initramfs (for
multipath-tools-boot) and sysvinit scripts and systemd service files.

Closes: #827407

diff --git a/debian/extra/multipath-discover b/debian/extra/multipath-discover
new file mode 100755
index 000..89be2ab
--- /dev/null
+++ b/debian/extra/multipath-discover
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# This script is used to discover multipath devices at boot. It can be
+# called from both the init script as well as the native systemd
+# service.
+#
+
+if [ -f /etc/default/multipath-tools ]; then
+	. /etc/default/multipath-tools
+else
+	MULTIPATH_TOOLS_HW_HANDLERS=""
+	MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
+/sbin/modprobe -qa ${MULTIPATH_TOOLS_HW_HANDLERS}
+/sbin/modprobe -qa ${MULTIPATH_TOOLS_MODULES}
diff --git a/debian/initramfs/hooks b/debian/initramfs/hooks
index 4b60873..94cb7e8 100644
--- a/debian/initramfs/hooks
+++ b/debian/initramfs/hooks
@@ -17,6 +17,13 @@ fi
 
 . /usr/share/initramfs-tools/hook-functions
 
+if [ -f /etc/default/multipath-tools ]; then
+. /etc/default/multipath-tools
+else
+MULTIPATH_TOOLS_HW_HANDLERS=""
+MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
 add_bindings()
 {
   if [ -r /etc/multipath/bindings ]; then
@@ -48,7 +55,12 @@ add_udev_rules
 [ -r /etc/multipath.conf ] && cp /etc/multipath.conf $DESTDIR/etc/
 add_bindings
 
-for x in dm-multipath dm-service-time dm-round-robin dm-emc; do
-	manual_add_modules ${x}
+for x in $MULTIPATH_TOOLS_HW_HANDLERS; do
+manual_add_modules ${x}
+done
+for x in $MULTIPATH_TOOLS_MODULES; do
+manual_add_modules ${x}
 done
+echo "MULTIPATH_TOOLS_HW_HANDLERS=\"$MULTIPATH_TOOLS_HW_HANDLERS\"" > $DESTDIR/etc/multipath-tools-boot.conf
+echo "MULTIPATH_TOOLS_MODULES=\"$MULTIPATH_TOOLS_MODULES\"" >> $DESTDIR/etc/multipath-tools-boot.conf
 
diff --git a/debian/initramfs/init-top b/debian/initramfs/init-top
index 51637e9..ad27e83 100644
--- a/debian/initramfs/init-top
+++ b/debian/initramfs/init-top
@@ -26,10 +26,15 @@ verbose()
 
 maybe_break pre-multipath
 VERBOSITY=0
-HW_HANDLERS=""
+
+if [ -f /etc/multipath-tools-boot.conf ]; then
+. /etc/multipath-tools-boot.conf
+else
+MULTIPATH_TOOLS_HW_HANDLERS=""
+fi
 
 verbose && log_begin_msg "Loading multipath hardware handlers"
-for module in ${HW_HANDLERS}; do
+for module in ${MULTIPATH_TOOLS_HW_HANDLERS}; do
   if modprobe --syslog "$module"; then
 verbose && log_success_msg "loaded module ${module}."
   else
diff --git a/debian/initramfs/local-top b/debian/initramfs/local-top
index 33a6a29..99e4222 100644
--- a/debian/initramfs/local-top
+++ b/debian/initramfs/local-top
@@ -26,14 +26,19 @@ verbose()
 
 maybe_break pre-multipath
 VERBOSITY=0
-MP_MODULES="dm-multipath dm-emc"
 
 if [ ! -e /sbin/multipath ]; then
 	exit 0
 fi
 
+if [ -f /etc/multipath-tools-boot.conf ]; then
+. /etc/multipath-tools-boot.conf
+else
+MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
 verbose && log_begin_msg "Loading multipath modules"
-for module in ${MP_MODULES}; do
+for module in ${MULTIPATH_TOOLS_MODULES}; do
   if modprobe "$module"; then
 verbose && log_success_msg "loaded module ${module}."
   else
diff --git a/debian/multipath-tools-boot.init b/debian/multipath-tools-boot.init
index 4be4904..41c6d01 100755
--- a/debian/multipath-tools-boot.init
+++ b/debian/multipath-tools-boot.init
@@ -32,7 +32,7 @@ set -e
 case "$1" in
   start)
 	log_action_begin_msg "$DESC"
-	modprobe dm-multipath 2> /dev/null || true
+	/lib/multipath/multipath-discover 2> /dev/null || true
 	$DAEMON -v $VERBOSITY
 	log_action_end_msg $?
 	;;
diff --git a/debian/multipath-tools.default b/debian/multipath-tools.default
new file mode 100644
index 000..09bcb88
--- /dev/null
+++ b/debian/multipath-tools.default
@@ -0,0 +1,8 @@
+# defaults file for multipath-tools and multipath-tools-boot.
+
+# SCSI hardware handler kernel modules loaded at boot. These are loaded
+# before modules listed in MULTIPATH_TOOLS_MODULES.
+MULTIPATH_TOOLS_HW_HANDLERS="scsi_dh scsi_dh_alua scsi_dh_emc scsi_dh_rdac"
+
+# Multipath kernel modules loaded at boot.
+MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath dm-round-robin dm-service-time"
diff --git a/debian/rules b/debian/rules
index 8645d20..b

Bug#827923: multipath-tools: lintian errors and warnings

2016-06-22 Thread Andrew Patterson
On Wed, 22 Jun 2016 18:44:49 + Niels Thykier  wrote:
> On Wed, 22 Jun 2016 11:42:54 -0600 Andrew Patterson > 
>  wrote: > > Package: multipath-tools > >
Version: 0.6.1-3 > > > > Hi, > > Please keep me in CC for follow ups on
this if my reply is desired. > > > I get the following errors and
warning when running lintian > > 2.5.44~bpo8+1 on
multipath-tools_0.6.1-3: > > > > [...] > > W: multipath-tools:
maintscript-calls-ldconfig postinst > > E: multipath-tools:
package-must-activate-ldconfig-trigger > > lib/libmultipath.so.0 > > W:
multipath-tools: maintscript-calls-ldconfig postrm > > These 3 can
appear for legitimate reasons if: > > * The build was last done several
months ago, OR > * If done in jessie/jessie-backports (with the jessie
version of > debhelper) > > In the former case, a simple no-change
rebuild will generally fix the > issue (and indeed, the unstable package
does not appear to be affected[1]). > In the latter case, lintian is
simply testing for something that > applies to unstable but not for
jessie. For backports, you can avoid it > by using the debhelper in
jessie-backports; but it is not really necessary. > > Thanks, > ~Niels

You are correct, the build was done in jessie. After rebuilding in sid,
I get:

lintian multipath-tools_0.6.1-3_amd64.changes
W: multipath-tools: package-name-doesnt-match-sonames libmpathcmd0
libmpathpersist0 libmultipath0
W: multipath-tools: hardening-no-relro lib/libmpathpersist.so.0
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_in.3.gz
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_out.3.gz
W: multipath-tools: manpage-has-errors-from-man
usr/share/man/man5/multipath.conf.5.gz 239: warning [p 4, 0.0i]: can't
break line
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man8/mpathpersist.8.gz
W: multipath-tools: non-dev-pkg-with-shlib-symlink lib/libmpathcmd.so.0
lib/libmpathcmd.so
W: multipath-udeb udeb: package-name-doesnt-match-sonames libmultipath0
N: 5 tags overridden (1 error, 1 warning, 3 info)

Is it worth it to try and fix these warning? Or at least provide
overrides for them.

Andrew

-- 
Andrew Patterson
Hewlett-Packard Enterprise




Bug#827923: multipath-tools: lintian errors and warnings

2016-06-22 Thread Andrew Patterson
Package: multipath-tools
Version: 0.6.1-3

I get the following errors and warning when running lintian
2.5.44~bpo8+1 on multipath-tools_0.6.1-3:

$ lintian multipath-tools_0.6.1-3_amd64.changes
W: multipath-tools: package-name-doesnt-match-sonames libmpathcmd0
libmpathpersist0 libmultipath0
W: multipath-tools: hardening-no-relro lib/libmpathpersist.so.0
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_in.3.gz
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_out.3.gz
W: multipath-tools: manpage-has-errors-from-man
usr/share/man/man5/multipath.conf.5.gz 239: warning [p 4, 0.0i]: can't
break line
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man8/mpathpersist.8.gz
W: multipath-tools: non-dev-pkg-with-shlib-symlink lib/libmpathcmd.so.0
lib/libmpathcmd.so
W: multipath-tools: maintscript-calls-ldconfig postinst
E: multipath-tools: package-must-activate-ldconfig-trigger
lib/libmultipath.so.0
W: multipath-tools: maintscript-calls-ldconfig postrm
W: multipath-udeb udeb: package-name-doesnt-match-sonames libmultipath0
N: 5 tags overridden (1 error, 1 warning, 3 info)

And here is the output when using --no-overrides:

$ lintian -o multipath-tools_0.6.1-3_amd64.changes
E: multipath-tools-boot: systemd-no-service-for-init-rcS-script
multipath-tools-boot
W: multipath-tools: package-name-doesnt-match-sonames libmpathcmd0
libmpathpersist0 libmultipath0
W: multipath-tools: hardening-no-relro lib/libmpathpersist.so.0
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_in.3.gz
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man3/mpath_persistent_reserve_out.3.gz
W: multipath-tools: manpage-has-errors-from-man
usr/share/man/man5/multipath.conf.5.gz 239: warning [p 4, 0.0i]: can't
break line
W: multipath-tools: manpage-has-bad-whatis-entry
usr/share/man/man8/mpathpersist.8.gz
W: multipath-tools: non-dev-pkg-with-shlib-symlink
lib/libmpathpersist.so.0 lib/libmpathpersist.so
W: multipath-tools: non-dev-pkg-with-shlib-symlink lib/libmpathcmd.so.0
lib/libmpathcmd.so
W: multipath-tools: maintscript-calls-ldconfig postinst
E: multipath-tools: package-must-activate-ldconfig-trigger
lib/libmultipath.so.0
W: multipath-tools: maintscript-calls-ldconfig postrm
W: multipath-udeb udeb: package-name-doesnt-match-sonames libmultipath0

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827407: multipath-tools: initramfs should load same mp modules as sysvinit/sysmtemd files

2016-06-22 Thread Andrew Patterson
On Thu, 16 Jun 2016 22:13:10 +0530 Ritesh Raj Sarraf  wrote:
> On Thu, 2016-06-16 at 10:20 -0600, Andrew Patterson wrote: > > > Yes. That'd 
> be good. Perhaps the module loading could be
delegated to a > > separate > > > shell script. Such a script can then
be invoked in ExecStartPre for systemd, > > and > > > for shell
initscripts we'll just have to source it into the main initscript. > > >
> Yes. Although it may take some work to get it into the initramfs. > >
> > A good example will be the open-iscsi package. The (co)maintainer
has some > pretty neat scripts on similar housekeeping work. > > > >Â >
> > Do you want to submit a patch for this ? > > >Â > > > > > > > I
would be happy too. Give me a couple of days. > > Thanks.

Here is the promised patch.

-- 
Andrew Patterson
Hewlett-Packard Enterprise

commit 916211466614fe267b02a1b940bc6bedc33479ec
Author: Andrew Patterson 
Date:   Wed Jun 22 10:46:50 2016 -0600

Load common set of multipath-modules in initramfs, sysvinit, and systemd

Use variables in /etc/default/multipath-tools to load a common set
of multipath kernel modules and hardware handlers in the initramfs (for
multipath-tools-boot) and sysvinit scripts and systemd service files.

Closes: #827407

diff --git a/debian/extra/multipath-discover b/debian/extra/multipath-discover
new file mode 100755
index 000..89be2ab
--- /dev/null
+++ b/debian/extra/multipath-discover
@@ -0,0 +1,16 @@
+#!/bin/sh
+#
+# This script is used to discover multipath devices at boot. It can be
+# called from both the init script as well as the native systemd
+# service.
+#
+
+if [ -f /etc/default/multipath-tools ]; then
+	. /etc/default/multipath-tools
+else
+	MULTIPATH_TOOLS_HW_HANDLERS=""
+	MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
+/sbin/modprobe -qa ${MULTIPATH_TOOLS_HW_HANDLERS}
+/sbin/modprobe -qa ${MULTIPATH_TOOLS_MODULES}
diff --git a/debian/initramfs/hooks b/debian/initramfs/hooks
index 4b60873..94cb7e8 100644
--- a/debian/initramfs/hooks
+++ b/debian/initramfs/hooks
@@ -17,6 +17,13 @@ fi
 
 . /usr/share/initramfs-tools/hook-functions
 
+if [ -f /etc/default/multipath-tools ]; then
+. /etc/default/multipath-tools
+else
+MULTIPATH_TOOLS_HW_HANDLERS=""
+MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
 add_bindings()
 {
   if [ -r /etc/multipath/bindings ]; then
@@ -48,7 +55,12 @@ add_udev_rules
 [ -r /etc/multipath.conf ] && cp /etc/multipath.conf $DESTDIR/etc/
 add_bindings
 
-for x in dm-multipath dm-service-time dm-round-robin dm-emc; do
-	manual_add_modules ${x}
+for x in $MULTIPATH_TOOLS_HW_HANDLERS; do
+manual_add_modules ${x}
+done
+for x in $MULTIPATH_TOOLS_MODULES; do
+manual_add_modules ${x}
 done
+echo "MULTIPATH_TOOLS_HW_HANDLERS=\"$MULTIPATH_TOOLS_HW_HANDLERS\"" > $DESTDIR/etc/multipath-tools-boot.conf
+echo "MULTIPATH_TOOLS_MODULES=\"$MULTIPATH_TOOLS_MODULES\"" >> $DESTDIR/etc/multipath-tools-boot.conf
 
diff --git a/debian/initramfs/init-top b/debian/initramfs/init-top
index 51637e9..ad27e83 100644
--- a/debian/initramfs/init-top
+++ b/debian/initramfs/init-top
@@ -26,10 +26,15 @@ verbose()
 
 maybe_break pre-multipath
 VERBOSITY=0
-HW_HANDLERS=""
+
+if [ -f /etc/multipath-tools-boot.conf ]; then
+. /etc/multipath-tools-boot.conf
+else
+MULTIPATH_TOOLS_HW_HANDLERS=""
+fi
 
 verbose && log_begin_msg "Loading multipath hardware handlers"
-for module in ${HW_HANDLERS}; do
+for module in ${MULTIPATH_TOOLS_HW_HANDLERS}; do
   if modprobe --syslog "$module"; then
 verbose && log_success_msg "loaded module ${module}."
   else
diff --git a/debian/initramfs/local-top b/debian/initramfs/local-top
index 33a6a29..99e4222 100644
--- a/debian/initramfs/local-top
+++ b/debian/initramfs/local-top
@@ -26,14 +26,19 @@ verbose()
 
 maybe_break pre-multipath
 VERBOSITY=0
-MP_MODULES="dm-multipath dm-emc"
 
 if [ ! -e /sbin/multipath ]; then
 	exit 0
 fi
 
+if [ -f /etc/multipath-tools-boot.conf ]; then
+. /etc/multipath-tools-boot.conf
+else
+MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+fi
+
 verbose && log_begin_msg "Loading multipath modules"
-for module in ${MP_MODULES}; do
+for module in ${MULTIPATH_TOOLS_MODULES}; do
   if modprobe "$module"; then
 verbose && log_success_msg "loaded module ${module}."
   else
diff --git a/debian/multipath-tools-boot.init b/debian/multipath-tools-boot.init
index 4be4904..41c6d01 100755
--- a/debian/multipath-tools-boot.init
+++ b/debian/multipath-tools-boot.init
@@ -32,7 +32,7 @@ set -e
 case "$1" in
   start)
 	log_action_begin_msg "$DESC"
-	modprobe dm-multipath 2> /dev/null || true
+	/lib/multipath/multi

Bug#814363: Fix debian/watch for multipath-tools

2016-06-21 Thread Andrew Patterson
On Wed, 17 Feb 2016 10:45:01 -0700 Nicholas Luedtke  
wrote:
> tags 814363 + patch
> 
> On 02/11/2016 01:07 AM, Ritesh Raj Sarraf wrote:
> > On Wed, 2016-02-10 at 13:09 -0700, Nicholas Luedtke wrote:
> >> Package: multipath-tools
> >> Version: 0.5.0+git1.656f8865-1
> >>
> >>
> >> The watch file is broken.  Below is the error message received when
> >> using the watch file. Seems like the page is no longer accessible?
> >>
> >> -- Scanning for watchfiles in .
> >> -- Found watchfile in ./debian
> >> -- In debian/watch, processing watchfile line:
> >>
> >> http://christophe.varoqui.free.fr/multipath-tools/multipath-tools-([\
> >> d\.]*)\.tar\.bz2
> >> uscan warning: In watchfile debian/watch, reading webpage
> >>   http://christophe.varoqui.free.fr/multipath-tools/ failed: 403
> >> Forbidden
> >> -- Scan finished
> > 
> > I actually had a more hilarious result looking at the upstream link,
> > after your bug report.
> > 
> > “Your requested URL has been blocked as per the directions received
> > from Department of Telecommunications, Government of India. Please
> > contact administrator for more information.”
> > 
> > 
> > Joke aside, in case of multipath-tools, it shouldn't matter. The
> > upstream releases are very slow. And these days most of the management
> > work is done in the git repo.
> > 
> > If you want you can send me a patch tracking the git repo's tags.
> > 
> > 
> 
> Thats an even better result for sure!
> 
> Attached is a patch for the watch file, but using uscan I think there is
> a bug which processes '-' before '+' in version comparison...
> 
> -- 
> Nicholas Luedtke
> Linux for HP Helion OpenStack, Hewlett-Packard Enterprise

Shouldn't this point to upstream rather than debian. In other words --
http://git.opensvc.com/gitweb.cgi?p=multipath-tools/.git;a=tags

Andrew

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-20 Thread Andrew Patterson
On Thu, 16 Jun 2016 15:19:22 +0530 Ritesh Raj Sarraf  wrote:
> Control: tag -1 +pending
> 
> On Wed, 2016-06-15 at 13:03 -0600, Andrew Patterson wrote:
> > For most users this is not an issue. How many systems are going to use
> > that many LUNs?
> > 
> 
> I agree.
> 
> > One counter-example is using one or more FC LUNs per VM guest on a
> > host. In such a case, one can just modify /etc/sysctl.conf to increase
> > the setting. 
> 
> Yes. Most hypervisors may want this. But again, like you said, most users do 
> not
> map that many LUNs. And recommended practice is to have a VM backed by a file
> backend mostly.


OpenStack/cinder can (and is) be uses to provision VMs with networked
drives. It is pretty common for cloud applications.  I don't know what
a practical limit for the number of volume per VMs per host should be.


> 
> > We can run into problems when using
> > multipath-tools-boot. In this case, we use the default kernel settings
> > for aio-max-nr since multipath is run from the initramfs. I have
> > worked around this by modifying the script in
> > /usr/share/initramfs/scripts/local-top/multipath to temporarilly set
> > aio-max-nr to 1048576 while doing discovery.
> 
> 
> So, I'll just add this to README.Debian. There's not much that we can do 
> beyond
> documenting such behavior.

I will work on patch to have a user-configurable setting of aio-max-nr
in the initramfs. You can then decide if it is worth it to apply it.

In the meantime, the README.Debian change sounds fine to me.

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827412: multipath-tools: /dev/mapper device files not removed when partition is removed

2016-06-20 Thread Andrew Patterson
.d/73-special-net-names.rules
Reading rules file: /lib/udev/rules.d/73-usb-net-by-mac.rules
Reading rules file: /lib/udev/rules.d/75-net-description.rules
Reading rules file: /lib/udev/rules.d/75-probe_mtd.rules
Reading rules file: /lib/udev/rules.d/78-sound-card.rules
Reading rules file: /lib/udev/rules.d/80-debian-compat.rules
Reading rules file: /lib/udev/rules.d/80-drivers.rules
Reading rules file: /lib/udev/rules.d/80-ifupdown.rules
Reading rules file: /lib/udev/rules.d/80-net-setup-link.rules
Reading rules file: /lib/udev/rules.d/85-hdparm.rules
Reading rules file: /lib/udev/rules.d/85-hwclock.rules
Reading rules file: /lib/udev/rules.d/90-console-setup.rules
Reading rules file: /lib/udev/rules.d/99-systemd.rules
rules contain 49152 bytes tokens (4096 * 12 bytes), 14495 bytes strings
2224 strings (27633 bytes), 1511 de-duplicated (13852 bytes), 714 trie nodes 
used
IMPORT 'dmsetup_env 254 8' /lib/udev/rules.d/60-kpartx.rules:12
starting 'dmsetup_env 254 8'
'dmsetup_env 254 8'(out) 'DM_UUID=part1-mpath-360002ac0190428be'
'dmsetup_env 254 8'(out) 'DM_TABLE_STATE=LIVE'
'dmsetup_env 254 8'(out) 'DM_STATE=ACTIVE'
'dmsetup_env 254 8'(out) 'DM_NAME=mpatha-part1'
Process 'dmsetup_env 254 8' succeeded.
IMPORT 'kpartx_id 254 8 part1-mpath-360002ac0190428be' 
/lib/udev/rules.d/60-kpartx.rules:17
starting 'kpartx_id 254 8 part1-mpath-360002ac0190428be'
'kpartx_id 254 8 part1-mpath-360002ac0190428be'(out) 
'DM_MPATH=mpatha'
'kpartx_id 254 8 part1-mpath-360002ac0190428be'(out) 'DM_PART=1'
'kpartx_id 254 8 part1-mpath-360002ac0190428be'(out) 
'DM_TYPE=scsi'
'kpartx_id 254 8 part1-mpath-360002ac0190428be'(out) 
'DM_WWN=0xpatha'
Process 'kpartx_id 254 8 part1-mpath-360002ac0190428be' 
succeeded.
LINK 'disk/by-id/scsi-mpatha-part1' /lib/udev/rules.d/60-kpartx.rules:22
LINK 'disk/by-id/wwn-0xpatha-part1' /lib/udev/rules.d/60-kpartx.rules:30
update old name, '/dev/disk/by-id/dm-name-mpatha-part1' no longer belonging to 
'/devices/virtual/block/dm-8'
no reference left, remove '/dev/disk/by-id/dm-name-mpatha-part1'
update old name, 
'/dev/disk/by-id/dm-uuid-part1-mpath-360002ac0190428be' no 
longer belonging to '/devices/virtual/block/dm-8'
no reference left, remove 
'/dev/disk/by-id/dm-uuid-part1-mpath-360002ac0190428be'
update old name, '/dev/disk/by-uuid/f3cf40d9-e78e-425b-b88c-807535188ea8' no 
longer belonging to '/devices/virtual/block/dm-8'
no reference left, remove 
'/dev/disk/by-uuid/f3cf40d9-e78e-425b-b88c-807535188ea8'
update old name, '/dev/mapper/mpatha-part1' no longer belonging to 
'/devices/virtual/block/dm-8'
no reference left, remove '/dev/mapper/mpatha-part1'
handling device node '/dev/dm-8', devnum=b254:8, mode=0600, uid=0, gid=0
preserve already existing symlink '/dev/block/254:8' to '../dm-8'
found 'b254:8' claiming '/run/udev/links/\x2fdisk\x2fby-id\x2fscsi-mpatha-part1'
creating link '/dev/disk/by-id/scsi-mpatha-part1' to '/dev/dm-8'
preserve already existing symlink '/dev/disk/by-id/scsi-mpatha-part1' to 
'../../dm-8'
found 'b254:8' claiming '/run/udev/links/\x2fdisk\x2fby-id\x2fwwn-0xpatha-part1'
creating link '/dev/disk/by-id/wwn-0xpatha-part1' to '/dev/dm-8'
preserve already existing symlink '/dev/disk/by-id/wwn-0xpatha-part1' to 
'../../dm-8'
created db file '/run/udev/data/b254:8' for '/devices/virtual/block/dm-8'
.ID_FS_TYPE_NEW=
ACTION=remote
DEVLINKS=/dev/disk/by-id/wwn-0xpatha-part1 /dev/disk/by-id/scsi-mpatha-part1
DEVNAME=/dev/dm-8
DEVPATH=/devices/virtual/block/dm-8
DEVTYPE=disk
DM_MPATH=mpatha
DM_NAME=mpatha-part1
DM_PART=1
DM_STATE=ACTIVE
DM_TABLE_STATE=LIVE
DM_TYPE=scsi
DM_UUID=part1-mpath-360002ac0190428be
DM_WWN=0xpatha
ID_FS_TYPE=
MAJOR=254
MINOR=8
SUBSYSTEM=block
TAGS=:systemd:
USEC_INITIALIZED=363164888098
Unload module index
Unloaded link configuration context.

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827412: multipath-tools: /dev/mapper device files not removed when partition is removed

2016-06-16 Thread Andrew Patterson
On Thu, 16 Jun 2016 22:54:08 +0530 Ritesh Raj Sarraf  wrote:
> On Thu, 2016-06-16 at 22:47 +0530, Ritesh Raj Sarraf wrote:
> > On Thu, 2016-06-16 at 22:35 +0530, Ritesh Raj Sarraf wrote:
> > > Control: tag -1 +pending
> > >Â 
> > > On Thu, 2016-06-16 at 10:48 -0600, Andrew Patterson wrote:
> > > > On Thu, 16 Jun 2016 16:15:54 +0530 Ritesh Raj Sarraf 
> > wrote:
> > > > > Control: tag -1 +confirmed
> > > > >Â 
> > > > >Â 
> > > > > Can you please try the attached patch ?
> > > > >Â 
> > > > >Â 
> > > > >Â 
> > > >Â 
> > > > That works:
> > >Â 
> > > Thank you very much. Will be part of next upload
> > 
> > Actually, my root cause may not be correct here. mpath*-* results in a wider
> > wildcard. The current one, mpath-* should have covered the mpath-WWID case.
> > But
> > that is what the bug is about.
> > 
> > Needs more investigation.
> 
> Can you please try it without the friendly names setting ?
> The current rule seems to be working fine in an environment without friendly
> names.
> 
> And please share output of /dev/disk/by-id/ with and without friendly names 
> set.
> 

With user_friendly_names turned off:

# ls -l /dev/mapper
total 0
lrwxrwxrwx 1 root root   7 Jun 16 12:42 360002ac0190428be 
-> ../dm-0
lrwxrwxrwx 1 root root   7 Jun 16 12:42 
360002ac0190428be-part1 -> ../dm-8
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190528be 
-> ../dm-1
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190628be 
-> ../dm-2
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190728be 
-> ../dm-3
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190828be 
-> ../dm-4
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190928be 
-> ../dm-5
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190a28be 
-> ../dm-6
lrwxrwxrwx 1 root root   7 Jun 16 12:38 360002ac0190b28be 
-> ../dm-7
crw--- 1 root root 10, 236 Jun 16 10:23 control

# ls -l /dev/disk/by-id
total 0
lrwxrwxrwx 1 root root 10 Jun 16 12:42 
dm-name-360002ac0190428be -> ../../dm-0
lrwxrwxrwx 1 root root 10 Jun 16 12:42 
dm-name-360002ac0190428be-part1 -> ../../dm-8
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190528be -> ../../dm-1
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190628be -> ../../dm-2
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190728be -> ../../dm-3
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190828be -> ../../dm-4
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190928be -> ../../dm-5
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190a28be -> ../../dm-6
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-name-360002ac0190b28be -> ../../dm-7
lrwxrwxrwx 1 root root 10 Jun 16 12:42 
dm-uuid-mpath-360002ac0190428be -> ../../dm-0
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190528be -> ../../dm-1
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190628be -> ../../dm-2
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190728be -> ../../dm-3
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190828be -> ../../dm-4
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190928be -> ../../dm-5
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190a28be -> ../../dm-6
lrwxrwxrwx 1 root root 10 Jun 16 12:38 
dm-uuid-mpath-360002ac0190b28be -> ../../dm-7
lrwxrwxrwx 1 root root 10 Jun 16 12:42 
dm-uuid-part1-mpath-360002ac0190428be -> ../../dm-8
lrwxrwxrwx 1 root root  9 Jun 16 10:23 scsi-0HP_LOGICAL_VOLUME_ -> 
../../sda
lrwxrwxrwx 1 root root 10 Jun 16 10:23 scsi-0HP_LOGICAL_VOLUME_-part1 
-> ../../sda1
lrwxrwxrwx 1 root root 10 Jun 16 10:23 scsi-0HP_LOGICAL_VOLUME_-part2 
-> ../../sda2
lrwxrwxrwx 1 root root 10 Jun 16 10:23 scsi-0HP_LOGICAL_VOLUME_-part3 
-> ../../sda3
lrwxrwxrwx 1 root root 10 Jun 16 12:42 scsi-360002ac0190428be 
-> ../../dm-0
lrwxrwxrwx 1 root root 10 Jun 16 12:42 
scsi-360002ac0190428be-part1 -> ../../dm-8
lrwxrwxrwx 1 root root 10 Jun 16 12:38 scsi-360002ac0190528be 
-> ../../dm-1
lrwxrwxrwx 1 root root 10 Jun 16 12:38 scsi-360002ac0190628be 
-> ../../dm-2
lrwxrwxrwx 1 root root 10 Jun

Bug#827412: multipath-tools: /dev/mapper device files not removed when partition is removed

2016-06-16 Thread Andrew Patterson
On Thu, 16 Jun 2016 16:15:54 +0530 Ritesh Raj Sarraf  wrote:
> Control: tag -1 +confirmed
> 
> 
> Can you please try the attached patch ?
> 
> 
> 

That works:

# ls -l /dev/mapper
total 0
crw--- 1 root root 10, 236 Jun 16 10:23 control
lrwxrwxrwx 1 root root   7 Jun 16 10:39 mpatha -> ../dm-0
lrwxrwxrwx 1 root root   7 Jun 16 10:39 mpatha-part1 -> ../dm-8
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathb -> ../dm-1
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathc -> ../dm-2
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathd -> ../dm-3
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathe -> ../dm-4
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathf -> ../dm-5
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathg -> ../dm-6
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathh -> ../dm-7


# fdisk /dev/mapper/mpatha

Welcome to fdisk (util-linux 2.28).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Invalid argument

The kernel still uses the old table. The new table will be used at the next 
reboot or after you run partprobe(8) or kpartx(8).

# ls -l /dev/mapper
total 0
crw--- 1 root root 10, 236 Jun 16 10:23 control
lrwxrwxrwx 1 root root   7 Jun 16 10:41 mpatha -> ../dm-0
lrwxrwxrwx 1 root root   7 Jun 16 10:44 mpathb -> ../dm-1
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathc -> ../dm-2
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathd -> ../dm-3
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathe -> ../dm-4
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathf -> ../dm-5
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathg -> ../dm-6
lrwxrwxrwx 1 root root   7 Jun 16 10:29 mpathh -> ../dm-7


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827407: multipath-tools: initramfs should load same mp modules as sysvinit/sysmtemd files

2016-06-16 Thread Andrew Patterson
On Thu, 16 Jun 2016 15:28:42 +0530 Ritesh Raj Sarraf  wrote:
> On Wed, 2016-06-15 at 14:05 -0600, Andrew Patterson wrote:
> > The systemd multpath-tools.service loads multipath modules using the 
> > following
> > line:
> > 
> > ExecStartPre=/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac dm-
> > multipath
> > 
> > While the sysvinit file /etc/init.d/multipath-tools-boot loads modules with:
> > 
> > modprobe dm-multipath 2> /dev/null || true
> > 
> > The /usr/share/initramfs-tools/scripts/local-top/multipath load modules 
> > using:
> > 
> > MP_MODULES="dm-multipath dm-emc"
> > for module in ${MP_MODULES}; do
> > Â  if modprobe "$module"; then
> > Â Â Â  verbose && log_success_msg "loaded module ${module}."
> > Â  else
> > Â Â Â  log_failure_msg "failed to load module ${module}."
> > Â  fi
> > done
> > 
> > All three of these methods should load the same set of modules (the
> > systemd one seems to be best).  Perhaps, there should be a
> > /etc/default/multipath-tools config file that governs which modules to
> > load?
> 
> Yes. That'd be good. Perhaps the module loading could be delegated to a 
> separate
> shell script. Such a script can then be invoked in ExecStartPre for systemd, 
> and
> for shell initscripts we'll just have to source it into the main initscript.

Yes. Although it may take some work to get it into the initramfs.

> 
> Do you want to submit a patch for this ?
> 
>

I would be happy too. Give me a couple of days.

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827412: multipath-tools: /dev/mapper device files not removed when partition is removed

2016-06-15 Thread Andrew Patterson
v/mapper/mpathb
GNU Parted 3.2
Using /dev/mapper/mpathb
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) rm 1
(parted) quit 
Information: You may need to update /etc/fstab.

# ls -l /dev/mapper/mpathb*
lrwxrwxrwx 1 root root 7 Jun 15 15:22 /dev/mapper/mpathb -> ../dm-3
# ls -l /dev/disk/by-id/scsi-mpathb*
lrwxrwxrwx 1 root root 10 Jun 15 15:22 /dev/disk/by-id/scsi-mpathb -> ../../dm-3

Here is the output for udevadm monitor when deleting the parted partition:

KERNEL[735.524037] change   /devices/virtual/block/dm-9 (block)
KERNEL[735.524470] change   /devices/virtual/block/dm-3 (block)
UDEV  [735.540771] change   /devices/virtual/block/dm-9 (block)
KERNEL[735.543465] add  /devices/virtual/bdi/254:10 (bdi)
KERNEL[735.543534] add  /devices/virtual/block/dm-10 (block)
UDEV  [735.543727] add  /devices/virtual/bdi/254:10 (bdi)
KERNEL[735.543795] remove   /devices/virtual/block/dm-10 (block)
UDEV  [735.548136] add  /devices/virtual/block/dm-10 (block)
UDEV  [735.548265] remove   /devices/virtual/block/dm-10 (block)
KERNEL[735.556226] remove   /devices/virtual/bdi/254:10 (bdi)
UDEV  [735.556447] remove   /devices/virtual/bdi/254:10 (bdi)
UDEV  [735.573117] change   /devices/virtual/block/dm-3 (block)
KERNEL[739.475199] change   /devices/virtual/block/dm-9 (block)
UDEV  [739.517662] change   /devices/virtual/block/dm-9 (block)
KERNEL[739.518452] remove   /devices/virtual/block/dm-9 (block)
KERNEL[739.518643] remove   /devices/virtual/block/dm-9 (block)
UDEV  [739.521683] remove   /devices/virtual/block/dm-9 (block)
UDEV  [739.521786] remove   /devices/virtual/block/dm-9 (block)
KERNEL[739.532256] remove   /devices/virtual/bdi/254:9 (bdi)
UDEV  [739.532847] remove   /devices/virtual/bdi/254:9 (bdi)
KERNEL[744.961662] change   /devices/virtual/block/dm-3 (block)
UDEV  [744.981294] change   /devices/virtual/block/dm-3 (block)

Note, we are now getting events for the partition as well as the parent
block device.

Note that when using parted, we get a different partition device file
-- /dev/mapper/mpathb1 than that created with fdisk. This difference
is discussed in bug
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=827308/

I have seen this same issue in earlier versions of multipath-tools.

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827407: multipath-tools: initramfs should load same mp modules as sysvinit/sysmtemd files

2016-06-15 Thread Andrew Patterson
Package: multipath-tools
Version: 0.6.1-3

The systemd multpath-tools.service loads multipath modules using the following
line:

ExecStartPre=/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac 
dm-multipath

While the sysvinit file /etc/init.d/multipath-tools-boot loads modules with:

modprobe dm-multipath 2> /dev/null || true

The /usr/share/initramfs-tools/scripts/local-top/multipath load modules using:

MP_MODULES="dm-multipath dm-emc"
for module in ${MP_MODULES}; do
  if modprobe "$module"; then
verbose && log_success_msg "loaded module ${module}."
  else
log_failure_msg "failed to load module ${module}."
  fi
done

All three of these methods should load the same set of modules (the
systemd one seems to be best).  Perhaps, there should be a
/etc/default/multipath-tools config file that governs which modules to
load?


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-15 Thread Andrew Patterson
On Thu, 16 Jun 2016 00:15:37 +0530 Ritesh Raj Sarraf  wrote:
> On Wed, 2016-06-15 at 11:12 -0600, Andrew Patterson wrote:
> > 
> > Thanks for reminding me about max_fds. At boot:
> > 
> > # cat /proc/sys/fs/nr_open
> > 1048576
> > 
> > and
> > 
> > # multipath -t | grep max_fds
> > Â Â Â Â Â Â Â  max_fds "max"
> > 
> > So this should already be set to 1048576. I tried the test again after
> > setting max_fd 1048576 in /etc/multipath.conf and got the same
> > behavior.
> 
> This dates way back. 2009 is a memorable year.
> 
> 
> https://www.redhat.com/archives/dm-devel/2009-July/msg00020.html
> 
> Quoting:
> 
> +
> +4. I see a lot of "io_setup failed" message using the directio checker
> +==
> +
> +The directio path checker makes use of the asynchronous I/O API (aio) 
> provided
> +by modern Linux systems. Asynchronous I/O allows an application to submit I/O
> +requests asynchronously and be notified later of their completion status. To
> +allow this, we must allocate an asynchronous I/O context (an object of type
> +aio_context_t) and this task is handled by the io_setup system call.
> +
> +A system wide limit on the number of AIO contexts that may be active
> +simultaneously is imposed via the aio-max-nr sysctl parameter.
> +
> +Once this limit has been reached further calls to io_setup will fail with the
> +error number EAGAIN leading to the "io_setup failed" messages seen for e.g.
> when
> +running "multipath -ll".
> +
> +To avoid this problem the number of available aio contexts should be 
> increased
> +by setting the aio-max-nr parameter. This can be set on a one-time basis via
> the
> +/proc file system, for e.g.:
> +
> +Â Â Â Â Â Â Â Â # echo 131072 > /proc/sys/fs/aio-max-nr
> +
> +Doubles the number of available contexts from the default value of 65536.
> +
> +To make this setting persistent a line may be added to /etc/sysctl.conf:
> +
> +Â Â Â Â Â Â Â Â fs.aio-max-nr = 131072
> +
> +Consult appropriate application and operating system tuning recommendations 
> for
> +guidance on appropriate values for this parameter.
> 
> 
> -- 
> Ritesh Raj Sarraf | http://people.debian.org/~rrs

For most users this is not an issue. How many systems are going to use
that many LUNs?

One counter-example is using one or more FC LUNs per VM guest on a
host. In such a case, one can just modify /etc/sysctl.conf to increase
the setting. We can run into problems when using
multipath-tools-boot. In this case, we use the default kernel settings
for aio-max-nr since multipath is run from the initramfs. I have
worked around this by modifying the script in
/usr/share/initramfs/scripts/local-top/multipath to temporarilly set
aio-max-nr to 1048576 while doing discovery.


-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827308: multipath-tools: wrong device file created when creating partitions with parted

2016-06-15 Thread Andrew Patterson

On 06/15/2016 08:26 AM, Ritesh Raj Sarraf wrote:
> Thank you for testing the new version. FYI, I've pushed it to Unstable. So, in
> all probability, this is the version that'll be shipped with Stretch.
> 
> 
> On Tue, 2016-06-14 at 14:13 -0600, Andrew Patterson wrote:
>> Package: multipath-tools
>> Version: 0.6.1-3
>>
>> When using parted to create partitions, multipath partition device
>> files are created using kernel names rather than honoring the "-p
>> -part" option in the various udev rules. This issue exists in
>> multipath-tools/kpartx_0.6.1-3, but has been seen in earlier versions
>> of multipath-tools/kpartx as well.
>>
> 
> 
> Okay!! But what is the expected behavior ? To the best of my knowledge, the
> naming convention is specific to particular storage layer. In case of 
> multipath,
> we delegate it to kpartx.

Yes, I would expect it to use the udev rules, e.g., mpatha-part1, not mpatha1.

> 
> 
>>  
>> # ls -l /dev/mapper
>> total 0
>> crw--- 1 root root 10, 236 Jun 14 13:28 control
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpatha -> ../dm-0
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathb -> ../dm-1
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathc -> ../dm-2
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathd -> ../dm-3
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathe -> ../dm-4
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathf -> ../dm-5
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathg -> ../dm-6
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathh -> ../dm-7
>>
>>
>> We then run parted on /dev/mapper/mpatha and create a label and a partition:
>>
>> # parted /dev/mapper/mpatha
>> GNU Parted 3.2
>> Using /dev/mapper/mpatha
>> Welcome to GNU Parted! Type 'help' to view a list of commands.
>> (parted) mklabel gptWarning: The existing disk label on /dev/mapper/mpatha
>> will be destroyed and all
>> data on this disk will be lost. Do you want to continue?
>> Yes/No? yes
>> (parted) mkpart primary ext2 0 100%
>> Warning: The resulting partition is not properly aligned for best 
>> performance.
>> Ignore/Cancel? i
>> (parted) quit
>> Information: You may need to update /etc/fstab.
>>
>> Under /dev/mapper, we see:
>>
>> # ls -l /dev/mapper
>> total 0
>> crw--- 1 root root 10, 236 Jun 14 13:28 control
>> lrwxrwxrwx 1 root root   7 Jun 14 13:57 mpatha -> ../dm-0
>> lrwxrwxrwx 1 root root   7 Jun 14 13:56 mpatha1 -> ../dm-8
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathb -> ../dm-1
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathc -> ../dm-2
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathd -> ../dm-3
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathe -> ../dm-4
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathf -> ../dm-5
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathg -> ../dm-6
>> lrwxrwxrwx 1 root root   7 Jun 14 13:53 mpathh -> ../dm-7
>>
>> Note that we see /dev/mapper/mpatha1 instead of /dev/mapper/mpatha-part1.
>>
>> Here is the udevadm monitor output during the partition creation:
>>
>> KERNEL[1654.655670] change   /devices/virtual/block/dm-0 (block)
>> UDEV  [1654.674470] change   /devices/virtual/block/dm-0 (block)
>> KERNEL[1686.963300] add  /devices/virtual/bdi/254:8 (bdi)
>> KERNEL[1686.963443] add  /devices/virtual/block/dm-8 (block)
>> UDEV  [1686.963889] add  /devices/virtual/bdi/254:8 (bdi)
>> KERNEL[1686.963916] change   /devices/virtual/block/dm-8 (block)
>> UDEV  [1686.972372] add  /devices/virtual/block/dm-8 (block)
>> UDEV  [1686.999211] change   /devices/virtual/block/dm-8 (block)
>> KERNEL[1699.598572] change   /devices/virtual/block/dm-0 (block)
>> KERNEL[1699.616084] add  /devices/virtual/bdi/254:9 (bdi)
>> KERNEL[1699.616215] add  /devices/virtual/block/dm-9 (block)
>> KERNEL[1699.616533] remove   /devices/virtual/block/dm-9 (block)
>> UDEV  [1699.616728] add  /devices/virtual/bdi/254:9 (bdi)
>> UDEV  [1699.620321] add  /devices/virtual/block/dm-9 (block)
>> UDEV  [1699.620427] remove   /devices/virtual/block/dm-9 (block)
>> KERNEL[1699.640199] remove   /devices/virtual/bdi/254:9 (bdi)
>> UDEV  [1699.640289] remove   /devices/virtual/bdi/254:9 (bdi)
>> UDEV  [1699.664986] change   /devices/virtual/block/dm-0 (block)
>>
> 
> 
> Do you know what these "remove" events are of ?

Not offhand. I can try udevadm test to perhaps figure it out.

> 
> 
>> This work 

Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-15 Thread Andrew Patterson
On 06/15/2016 08:43 AM, Ritesh Raj Sarraf wrote:
> Control: tag -1 moreinfo
> 
> On Tue, 2016-06-14 at 17:30 -0600, Andrew Patterson wrote:
>> There is no output in dmesg.
>>
>> Changing /proc/sys/fs/aio-max-nr from the default 65536 to 1048576
>> seems to help. With this change, I can go up to 256 LUNs (1024 paths)
>> with no problems.
>>
>> I have seen similar issues in earlier versions of multipath-tools.
> 
> Thank you for bringing this up, and the pointer. In multipath.conf there was a
> directive added for something similar.
> 
> 
>max_fds  Specify  the  maximum  number of file descriptors that
> can be opened by multipath and  multipathd.   This  is
> equivalent  to ulimit -n. A value of max will set this
> to the system limit from /proc/sys/fs/nr_open. If this
> is  not  set,  the maximum number of open fds is taken
> from the calling process. It is usually  1024.  To  be
> safe,  this  should  be  set  to the maximum number of
> paths plus 32, if that number is greated than 1024.
> 
> 
> It seems to be inline with what aio-max-nr documentation says.
> 
> ==
> 
> aio-nr & aio-max-nr:
> 
> aio-nr is the running total of the number of events specified on the
> io_setup system call for all currently active aio contexts.  If aio-nr
> reaches aio-max-nr then io_setup will fail with EAGAIN.  Note that
> raising aio-max-nr does not result in the pre-allocation or re-sizing
> of any kernel data structures.
> 
> ==
> 
> 
> Have you tried it with the "max_fds" option set ?
> 
> 

Thanks for reminding me about max_fds. At boot:

# cat /proc/sys/fs/nr_open
1048576

and

# multipath -t | grep max_fds
max_fds "max"

So this should already be set to 1048576. I tried the test again after
setting max_fd 1048576 in /etc/multipath.conf and got the same
behavior.


--
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827322: multipath-tools: Not all paths discovered when large number of paths are present

2016-06-14 Thread Andrew Patterson
proc/sys/fs/aio-max-nr from the default 65536 to 1048576
seems to help. With this change, I can go up to 256 LUNs (1024 paths)
with no problems.

I have seen similar issues in earlier versions of multipath-tools.

--
Andrew Patterson
Hewlett-Packard Enterprise



Bug#827308: multipath-tools: wrong device file created when creating partitions with parted

2016-06-14 Thread Andrew Patterson
Package: multipath-tools
Version: 0.6.1-3

When using parted to create partitions, multipath partition device
files are created using kernel names rather than honoring the "-p
-part" option in the various udev rules. This issue exists in
multipath-tools/kpartx_0.6.1-3, but has been seen in earlier versions
of multipath-tools/kpartx as well.

Example:

Testing on an amd64 system running sid with
multipath-tools/kpartx_0.6.1-3 from experimental. We hav 8 FC LUNs
with 4 paths each. The kernel version is 4.6.0-1-amd64.

With the following /etc/multipath.conf file:

defaults {
user_friendly_names "yes"
}

blacklist {
device {
vendor "HP"
product "LOGICAL VOLUME"
}

Here are the devices used (the boot device is blacklisted):

# lsscsi
[0:0:0:0]storage HP   P244br   2.52  -
[0:1:0:0]diskHP   LOGICAL VOLUME   2.52  /dev/sda 
[1:0:0:0]disk3PARdata VV   3212  /dev/sdj 
[1:0:0:1]disk3PARdata VV   3212  /dev/sdl 
[1:0:0:2]disk3PARdata VV   3212  /dev/sdn 
[1:0:0:3]disk3PARdata VV   3212  /dev/sdq 
[1:0:0:4]disk3PARdata VV   3212  /dev/sdr 
[1:0:0:5]disk3PARdata VV   3212  /dev/sdt 
[1:0:0:6]disk3PARdata VV   3212  /dev/sdv 
[1:0:0:7]disk3PARdata VV   3212  /dev/sdx 
[1:0:0:254]  enclosu 3PARdata SES  3212  -
[1:0:1:0]disk3PARdata VV   3212  /dev/sdz 
[1:0:1:1]disk3PARdata VV   3212  /dev/sdaa
[1:0:1:2]disk3PARdata VV   3212  /dev/sdab
[1:0:1:3]disk3PARdata VV   3212  /dev/sdac
[1:0:1:4]disk3PARdata VV   3212  /dev/sdad
[1:0:1:5]disk3PARdata VV   3212  /dev/sdae
[1:0:1:6]disk3PARdata VV   3212  /dev/sdaf
[1:0:1:7]disk3PARdata VV   3212  /dev/sdag
[1:0:1:254]  enclosu 3PARdata SES  3212  -
[2:0:0:0]disk3PARdata VV   3212  /dev/sdb 
[2:0:0:1]disk3PARdata VV   3212  /dev/sdc 
[2:0:0:2]disk3PARdata VV   3212  /dev/sdd 
[2:0:0:3]disk3PARdata VV   3212  /dev/sde 
[2:0:0:4]disk3PARdata VV   3212  /dev/sdf 
[2:0:0:5]disk3PARdata VV   3212  /dev/sdg 
[2:0:0:6]disk3PARdata VV   3212  /dev/sdh 
[2:0:0:7]disk3PARdata VV   3212  /dev/sdi 
[2:0:0:254]  enclosu 3PARdata SES  3212  -
[2:0:1:0]disk3PARdata VV   3212  /dev/sdk 
[2:0:1:1]disk3PARdata VV   3212  /dev/sdm 
[2:0:1:2]disk3PARdata VV   3212  /dev/sdo 
[2:0:1:3]disk3PARdata VV   3212  /dev/sdp 
[2:0:1:4]disk3PARdata VV   3212  /dev/sds 
[2:0:1:5]disk3PARdata VV   3212  /dev/sdu 
[2:0:1:6]disk3PARdata VV   3212  /dev/sdw 
[2:0:1:7]disk3PARdata VV   3212  /dev/sdy 
[2:0:1:254]  enclosu 3PARdata SES  3212  - 

And the following multipath -l output:

# multipath -l
mpathe (360002ac0190828be) dm-4 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  |- 1:0:0:4 sdr  65:16  active undef running
  |- 1:0:1:4 sdad 65:208 active undef running
  |- 2:0:0:4 sdf  8:80   active undef running
  `- 2:0:1:4 sds  65:32  active undef running
mpathd (360002ac0190728be) dm-3 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  |- 1:0:0:3 sdq  65:0   active undef running
  |- 1:0:1:3 sdac 65:192 active undef running
  |- 2:0:0:3 sde  8:64   active undef running
  `- 2:0:1:3 sdp  8:240  active undef running
mpathc (360002ac0190628be) dm-2 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  |- 1:0:0:2 sdn  8:208  active undef running
  |- 1:0:1:2 sdab 65:176 active undef running
  |- 2:0:0:2 sdd  8:48   active undef running
  `- 2:0:1:2 sdo  8:224  active undef running
mpathb (360002ac0190528be) dm-1 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  |- 1:0:0:1 sdl  8:176  active undef running
  |- 1:0:1:1 sdaa 65:160 active undef running
  |- 2:0:0:1 sdc  8:32   active undef running
  `- 2:0:1:1 sdm  8:192  active undef running
mpatha (360002ac0190428be) dm-0 3PARdata,VV
size=1.0G features='0' hwhandler='0' wp=rw
`-+- policy='service-time 0' prio=0 status=active
  |- 1:0:0:0 sdj  8:144  active undef running
  |- 1:0:1:0 sdz  65:144 active undef running
  |- 2:0:0:0 sdb  8:16   active undef running
  `- 2:0:1:0 sdk  8:160  active undef running
mpathh (360002ac0190b28be) dm-7 3PARdata,VV
size=1.0G features='0' h

Bug#821049: multipath-tools: reloading devmap results in block device being created incorrectly

2016-04-17 Thread Andrew Patterson
On Fri, 15 Apr 2016 16:07:48 -0600 Andrew Patterson
 wrote:
> On Sat, 16 Apr 2016 00:42:33 +0530 Ritesh Raj Sarraf  wrote: 
> > > On Sat, 2016-04-16 at 00:35 +0530, Ritesh Raj Sarraf wrote: > >
Hello Mike, > > > > All I can say is that I am able to see the >
behavior that you've mentioned (So > > marking the bug as confirmed). >
THis is seen on my LIO iSCSI Setup running on 2 > > Guest VMs. > > > > >
In doing that, I was surprised that those dev/mapper/ entries are now >
> > symlinks. > > My storage knowledge has rusted for some time now, but
> from what I recollect, > > it > > used to by direct block node names,
> not symlinks. > > Maybe this could be of some help to you in debugging
> this further. > > Apr 16 00:34:11 debian-sanboot systemd[1]: Started >
Device-Mapper Multipath > Device Controller. > Apr 16 00:34:11 >
debian-sanboot systemd-udevd[949]: conflicting device node > >
'/dev/mapper/sanroot' found, link to '/dev/dm-0' will not be created > >
> > So I think my understanding was correct about Device Mapper's >
behavior. Now I'm > not sure what 'systemd-udevd[949]' is reflecting >
here as. I doubt if that is the > ID for the rules processing because >
then I'd have seen the respective rule. > > I hope this is of some use >
to you. My Debian time for today is running out :-) > > > Hi Ritesh, > >
I am working with Mike on this problem. > > We have seen this issue
using multiple transports (FCoE, FC, and > iSCSI), although most of our
testing has been using QLogic FC cards > with 3Par FC storage. I don't
think this is a transport problem, but > is instead, some sort of
udev/multipath interaction problem. > > Here is a test I ran with
multipath-tools_0.5.0-7 and > multipath-tools_0.5.0+git0.770e6d0d-1 on
stretch using the same > hardware and configuration that Mike has been
using: > > Here is the storage: > > 1:0:0:1] disk 3PARdata VV 3212
/dev/sdb > [1:0:0:2] disk 3PARdata VV 3212 /dev/sdc > [1:0:0:254]
enclosu 3PARdata SES 3212 - > [1:0:1:1] disk 3PARdata VV 3212 /dev/sdd >
[1:0:1:2] disk 3PARdata VV 3212 /dev/sde > [1:0:1:254] enclosu 3PARdata
SES 3212 - > > And udev monitor output (udevadm monitor -u -k): > >
multipath-tools_0.5.0-7 > > # udevadm monitor -k -u > monitor will print
the received events for: > UDEV - the event which udev sends out after
rule processing > KERNEL - the kernel uevent > > KERNEL[90540.185998]
change /devices/virtual/block/dm-0 (block) > KERNEL[90540.187857] change
/devices/virtual/block/dm-1 (block) > UDEV [90540.265654] change
/devices/virtual/block/dm-1 (block) > UDEV [90540.266138] change
/devices/virtual/block/dm-0 (block) > > # ls -l /dev/mapper > total 0 >
crw--- 1 root root 10, 236 Apr 14 14:31 control > lrwxrwxrwx 1 root
root 7 Apr 15 15:40 mpatha -> ../dm-0 > lrwxrwxrwx 1 root root 7 Apr 15
15:40 mpathb -> ../dm-1 >

My git bisection resulted in this commit causing the change in behavior:

commit 4a2431aa944eb2e5b6f3ccd2d4fe1df67f9e5679
Author: Hannes Reinecke 
Date:   Tue Jul 29 15:44:46 2014 +0200

Fixup device-mapper 'cookie' handling
   
device-mapper has a 'cookie', which is inserted with the ioctl
for modifying device-mapper devices.
It is used as a synchronization point between udev and any other
applications to notify the latter when udev has finished
processing the event.
Originally multipath would only use a single cookie for every
transaction, and wait for that cookie at the end of the program.
Which works well if you only have one transaction, but for several
(like calling 'multipath') it will actually overwrite the cookie
and fail to wait for earlier events.
This causes libdevmapper to create the device nodes on its own,
and the device nodes not being handled by udev.
   
Signed-off-by: Hannes Reinecke 

So it looks like this is the designed behavior.

-- 
Andrew Patterson
Hewlett-Packard Enterprise



Bug#821049: multipath-tools: reloading devmap results in block device being created incorrectly

2016-04-15 Thread Andrew Patterson
On Sat, 16 Apr 2016 00:42:33 +0530 Ritesh Raj Sarraf  wrote:
> On Sat, 2016-04-16 at 00:35 +0530, Ritesh Raj Sarraf wrote: > > Hello Mike, > 
> > > > All I can say is that I am able to see the
behavior that you've mentioned (So > > marking the bug as confirmed).
THis is seen on my LIO iSCSI Setup running on 2 > > Guest VMs. > > > >
In doing that, I was surprised that those dev/mapper/ entries are now >
> symlinks. > > My storage knowledge has rusted for some time now, but
from what I recollect, > > it > > used to by direct block node names,
not symlinks. > > Maybe this could be of some help to you in debugging
this further. > > Apr 16 00:34:11 debian-sanboot systemd[1]: Started
Device-Mapper Multipath > Device Controller. > Apr 16 00:34:11
debian-sanboot systemd-udevd[949]: conflicting device node >
'/dev/mapper/sanroot' found, link to '/dev/dm-0' will not be created > >
> So I think my understanding was correct about Device Mapper's
behavior. Now I'm > not sure what 'systemd-udevd[949]' is reflecting
here as. I doubt if that is the > ID for the rules processing because
then I'd have seen the respective rule. > > I hope this is of some use
to you. My Debian time for today is running out :-) >

Hi Ritesh,

I am working with Mike on this problem.

We have seen this issue using multiple transports (FCoE, FC, and
iSCSI), although most of our testing has been using QLogic FC cards
with 3Par FC storage. I don't think this is a transport problem, but
is instead, some sort of udev/multipath interaction problem.

Here is a test I ran with multipath-tools_0.5.0-7 and
multipath-tools_0.5.0+git0.770e6d0d-1 on stretch using the same
hardware and configuration that Mike has been using:

Here is the storage:

1:0:0:1]disk3PARdata VV   3212  /dev/sdb
[1:0:0:2]disk3PARdata VV   3212  /dev/sdc
[1:0:0:254]  enclosu 3PARdata SES  3212  -   
[1:0:1:1]disk3PARdata VV   3212  /dev/sdd
[1:0:1:2]disk3PARdata VV   3212  /dev/sde
[1:0:1:254]  enclosu 3PARdata SES  3212  -   

And udev monitor output (udevadm monitor -u -k):

multipath-tools_0.5.0-7

# udevadm monitor -k -u
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

KERNEL[90540.185998] change   /devices/virtual/block/dm-0 (block)
KERNEL[90540.187857] change   /devices/virtual/block/dm-1 (block)
UDEV  [90540.265654] change   /devices/virtual/block/dm-1 (block)
UDEV  [90540.266138] change   /devices/virtual/block/dm-0 (block)

# ls -l /dev/mapper
total 0
crw--- 1 root root 10, 236 Apr 14 14:31 control
lrwxrwxrwx 1 root root   7 Apr 15 15:40 mpatha -> ../dm-0
lrwxrwxrwx 1 root root   7 Apr 15 15:40 mpathb -> ../dm-1

multipath-tools_0.5.0+git0.770e6d0d-1

# udevadm monitor -k -u
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

KERNEL[91354.622867] change   /devices/virtual/block/dm-0 (block)
KERNEL[91354.625218] change   /devices/virtual/block/dm-1 (block)
UDEV  [91354.699455] change   /devices/virtual/block/dm-0 (block)
UDEV  [91354.701663] change   /devices/virtual/block/dm-1 (block)

So the udev events look identical. We think that perhaps we have a
race and the new version of multipath is not waiting (or waiting long
enough) for udev to crate the device files and just goes and creates
the required files.

I am going to try and bisect the changes in the upstream multipath code
between
0.5.0 and 770e6d0d to see where this breaks.

This does not seem to that serious a problem on its own, since the
block device that is created is identical to what the symlink would
point to. But we are seeing other issue. But we are seeing other
issues like device files reappearing after the running multipath -f on
it, removing the LUN, and then running multipath -r which we think is
related.

-- 
Andrew Patterson
Hewlett-Packard Enterprise




Bug#778056: Patch for GCC 5 build issue

2015-07-08 Thread Andrew Patterson
tags 778056 + patch
thanks

Here's a fix for the GCC 5 build issue.  It adds 'export CFLAGS =
-std=gnu89' to the rules file to use the GNU89 inline semantics.

I tested it with both gcc-5.0 and gcc-4.9.

Note: we still get a lot of compiler warnings, but that is not a gcc-5 issue.

-- 
Andrew Patterson
Hewlett-Packard


--- pads-1.2/debian/rules.orig	2015-07-08 21:07:34.072665005 +
+++ pads-1.2/debian/rules	2015-07-08 21:08:22.601182891 +
@@ -10,7 +10,7 @@
 DEB_BUILD_GNU_TYPE  ?= $(shell dpkg-architecture -qDEB_BUILD_GNU_TYPE)
 
 
-CFLAGS = -Wall -g
+CFLAGS = -Wall -g -std=gnu89
 
 ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
 	CFLAGS += -O0


Bug#778055: Patch for GCC 5 build issue

2015-07-07 Thread Andrew Patterson
tags 778055 + patch
thanks

Here's a fix for the GCC 5 build issue.  It adds 'export CFLAGS =
-std=gnu89' to the rules file to use the GNU89 inline semantics.

I tested it with both gcc-5.0 and gcc-4.9.

-- 
Andrew Patterson
Hewlett-Packard

--- padevchooser-0.9.4/debian/rules.orig	2015-07-07 23:23:39.391825153 +
+++ padevchooser-0.9.4/debian/rules	2015-07-07 23:23:43.407882856 +
@@ -1,5 +1,7 @@
 #!/usr/bin/make -f
 
+export CFLAGS = -std=gnu89
+
 %:
 	dh $@ --with autoreconf --parallel
 



Bug#778055: Patch for GCC 5 build issue

2015-07-07 Thread Andrew Patterson
tags 778055 + patch

Here's a fix for the GCC 5 build issue.  It adds 'export CFLAGS =
-std=gnu89' to the rules file to use the GNU89 inline semantics.

I tested it with both gcc-5.0 and gcc-4.9.

-- 
Andrew Patterson
Hewlett-Packard

--- padevchooser-0.9.4/debian/rules.orig	2015-07-07 23:23:39.391825153 +
+++ padevchooser-0.9.4/debian/rules	2015-07-07 23:23:43.407882856 +
@@ -1,5 +1,7 @@
 #!/usr/bin/make -f
 
+export CFLAGS = -std=gnu89
+
 %:
 	dh $@ --with autoreconf --parallel
 



Bug#778055: Patch for GCC 5 build issue

2015-07-07 Thread Andrew Patterson
tags  + patch

Here's a fix for the GCC 5 build issue.  It adds 'export CFLAGS =
-std=gnu89' to the rules file to use the GNU89 inline semantics.

I tested it with both gcc-5.0 and gcc-4.9.

-- 
Andrew Patterson
Hewlett-Packard
--- padevchooser-0.9.4/debian/rules.orig	2015-07-07 23:23:39.391825153 +
+++ padevchooser-0.9.4/debian/rules	2015-07-07 23:23:43.407882856 +
@@ -1,5 +1,7 @@
 #!/usr/bin/make -f
 
+export CFLAGS = -std=gnu89
+
 %:
 	dh $@ --with autoreconf --parallel
 


Bug#764395: openvpn-auth-ldap: segmentation fault

2014-10-08 Thread Andrew Patterson
On Wed, 8 Oct 2014 18:40:40 +0200 Alberto Gonzalez Iniesta  
wrote:
> Hi Andrew,
>
> The plugin segfaults because it needs a config file as a parameter.
> It's not a nice behaviour, but not a critial one (since it needs a
> config file to be useful).
>
> Try:
>
> openvpn --dev null --plugin /usr/lib/openvpn/openvpn-auth-ldap.so ldap.cf
>
> Regards,
>
> Alberto
>

Yes, Specifying the config file fixed my segmentation fault issue. Should we 
open a new bug? Rename this one? Or consider this one closed?

Thanks,

-- Andrew


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#764395: openvpn-auth-ldap: segmentation fault

2014-10-07 Thread Andrew Patterson
Package: openvpn-auth-ldap
Version: 2.0.3-5.1
Tags: wheezy

The openvpn-auth plugin gets a segmentation fault at startup. This can
be demonstrated with the following command:

# openvpn --dev null --plugin /usr/lib/openvpn/openvpn-auth-ldap.so
Sat Oct  4 16:59:56 2014 OpenVPN 2.2.1 x86_64-linux-gnu [SSL] [LZO2] [EPOLL] 
[PKCS11] [eurephia] [MH] [PF_INET6] [IPv6 payload 20110424-2 (2.2RC2)] built on 
Jun 18 2013
Sat Oct  4 16:59:56 2014 IMPORTANT: OpenVPN's default port number is now 1194, 
based on an official port number assignment by IANA.  OpenVPN 2.0-beta16 and 
earlier used 5000 as the default port.
Sat Oct  4 16:59:56 2014 NOTE: OpenVPN 2.1 requires '--script-security 2' or 
higher to call user-defined scripts or executables
Segmentation fault (core dumped)

Note this issue seems to be a duplicate of:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=641811

The core dump shows:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `openvpn --dev null --plugin 
/usr/lib/openvpn/openvpn-auth-ldap.so'.
Program terminated with signal 11, Segmentation fault.
#0  0x7fa3ad91df21 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) bt
#0  0x7fa3ad91df21 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x7fa3ad259184 in ?? () from /usr/lib/openvpn/openvpn-auth-ldap.so
#2  0x7fa3ad25c38b in ?? () from /usr/lib/openvpn/openvpn-auth-ldap.so
#3  0x7fa3ad255ea7 in openvpn_plugin_open_v1 ()
   from /usr/lib/openvpn/openvpn-auth-ldap.so
#4  0x7fa3aeb2a214 in ?? ()
#5  0x7fa3aeafb1da in ?? ()
#6  0x7fa3aeafcbeb in ?? ()
#7  0x7fa3aeafd57f in ?? ()
#8  0x7fa3aeae8d14 in main ()

Details:

Debian wheezy 7.6.

ii  openvpn2.2.1-8+deb7 amd64virtual private network daemon
ii  openvpn-auth-l 2.0.3-5.1amd64OpenVPN LDAP authentication modul



-- Andrew


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#759740: apt-cache-ng: user apt-cacher-ng already exists error when upgrading

2014-09-04 Thread Andrew Patterson
>
> I don't know. And as I mentioned before, feel free to use the backport
> version. It contains a quick workaround for a similar situation:
>
># user should exist. adduser sometimes fails (system range issue) but 
> that's ok
>adduser --quiet --system --group --no-create-home --home "$CDIR" $NAME || 
> id $NAME
>
> Regards,
> Eduard.

We used your backports work-around and everything is working fine now.

-- Andrew



Bug#759740: apt-cache-ng: user apt-cacher-ng already exists error when upgrading

2014-09-03 Thread Andrew Patterson
On Wed, 3 Sep 2014 19:13:46 +0200 Eduard Bloch  wrote:
> reassign 759740 adduser
> retitle 759740 --system --quiet ... still produces warnings and bad exit code
> thanks
>
> Hallo,
> * Andrew Patterson [Tue, Sep 02 2014, 03:24:10PM]:
>
> > I guess I don't understand. The current account seem perfectly valid, i.e., 
> > uid = 108. Why is the configure scripts trying to re-add the user if it 
> > already exists? I can re-add the user, but what ID should I use?
> >
> > I tried running adduser by hand and got what I think are the expected 
> > results:
> >
> > # adduser --quiet --system --group --no-create-home --home 
> > /var/cache/apt-cacher-ng apt-cacher-ng
> > adduser: The user `apt-cacher-ng' already exists. Exiting.
>
> Nope, those are not the expected results:
>
> Add a system user
> If called with one non-option argument and the --system option, adduser will 
> add a
> system user. If a user with the same name already exists in the system uid 
> range
> (or, if the uid is specified, if a user with that uid already exists), 
> adduser will
> exit with a warning. This warning can be suppressed by adding "--quiet".
>
> Now I am also wondering what is going on. The last year's bug was clearly 
> reproducible with UID being outside of the system UID range. Your problem 
> apparently has a different reason, therefore I am reassigning this BR.
>
> > I am guessing I could delete the user, then rerun the configure script 
> > everything would fine.
>
> Yes and no... you would need to change the ownership of the the 
> logdir/cachedir contents (chown -R...) but appart from that it should do the 
> job.
>
> Thanks,
> Eduard.
>
>

I downloaded the deb source for apt-cacher-ng and found this in
apt-cacher-ng.postinst:

if [ "$1" = "configure" ]; then

   adduser --quiet --system --group --no-create-home --home $CDIR $NAME

   for x in $CDIR $LDIR ; do
  if [ ! -d "$x" ]; then
 install -d -g $NAME -o $NAME -m2755 "$x"
  fi
   done

So the script is not checking if the user exists before adding
it. Should configure be run when upgrading the package?

One possible complication I may be running into is that I am using
LDAP for user accounts with uid > 2000, which would fail if
apt-cacher-ng was trying to create a user in the range. But I think
the problem here is solely that the account already exists and the
code should be more like:


if [ "$1" = "configure" ]; then

   if getent passwd $NAME; then
  : # Or pehaps remove the user and re-add.
   else
  adduser --quiet --system --group --no-create-home --home $CDIR $NAME
   fi

-- Andrew



Bug#759740: apt-cache-ng: user apt-cacher-ng already exists error when upgrading

2014-09-02 Thread Andrew Patterson
On Sun, 31 Aug 2014 21:32:01 +0200 Eduard Bloch  wrote:
> Control: forcemerge 706382 759740
>
> > > I get the following error when trying to upgrade apt-cache-ng on wheezy:
> > >
> > > Setting up apt-cacher-ng (0.7.11-2) ...
> > > adduser: The user `apt-cacher-ng' already exists. Exiting.
> > > dpkg: error processing apt-cacher-ng (--configure):
> > > subprocess installed post-installation script returned error exit status 1
> > > Errors were encountered while processing:
> > > apt-cacher-ng
> > > E: Sub-process /usr/bin/dpkg returned an error code (1)
>
> Yeah, sorry, that was fixed back in 2013. Please use the backport version
> or revert your users UID to the range which is accepted by adduser (see
> the linked bugreport for details). That would also require a chown-R run
> on your cache and log directories, of course.
>
> Regards,
> Eduard.
>
>

I guess I don't understand. The current account seem perfectly valid, i.e., uid 
= 108. Why is the configure scripts trying to re-add the user if it already 
exists? I can re-add the user, but what ID should I use?

I tried running adduser by hand and got what I think are the expected results:

# adduser --quiet --system --group --no-create-home --home 
/var/cache/apt-cacher-ng apt-cacher-ng
adduser: The user `apt-cacher-ng' already exists. Exiting.

I am guessing I could delete the user, then rerun the configure script 
everything would fine.

-- Andrew


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#759740: apt-cache-ng: user apt-cacher-ng already exists error when upgrading

2014-08-29 Thread Andrew Patterson
Package: apt-cache-ng
Version: 0.7.11-2
Severity: important
Tags: wheezy

I get the following error when trying to upgrade apt-cache-ng on wheezy:

Setting up apt-cacher-ng (0.7.11-2) ...
adduser: The user `apt-cacher-ng' already exists. Exiting.
dpkg: error processing apt-cacher-ng (--configure):
 subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
 apt-cacher-ng
E: Sub-process /usr/bin/dpkg returned an error code (1)

And the user does indeed already exist (as you would expect with an
upgrade):

$ grep apt-cacher-ng /etc/passwd
apt-cacher-ng:x:0:108::/var/cache/apt-cacher-ng:/bin/false
acng:x:0:108::/var/cache/apt-cacher-ng:/bin/false

-- Andrew


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#759737: debian-installer: Need preseed entry to select default grub device

2014-08-29 Thread Andrew Patterson
Package: grub-installer
Version: 1.95
Severity: whishlist
Tags: jessie, patch

I am trying to create a completely automated preseed.cfg file for
a jessie installation that will work on several systems. In my
case, both QEMU VMs and physical servers. Both of these types of
systems have only one possible boot device but have different
boot device files, e,g, /dev/vda and /dev/sda.  The normal way to
do this would be to use "grub-installer/bootdev string" config option.
This option does not work here because we need to specify the exact
device -- /dev/sda or /dev/vda instead of the "only" device.

If I don't use "grub-installer/bootdev", the installer will ask
me whether I want to install on the device found or "nothing", so
my automation fails.

I developed a patch that lets you use the string "default" for
grub-installer/bootdev. This option always picks the appropriate
device. Example:

grub-installer/bootdev string default

Here is the patch:

Author: Andrew Patterson 
Date:   Thu Aug 14 14:00:47 2014 -0600

Using "default" for bootdev will install grub on default disk.

The grub-install/bootdev configuration value is used in a
file to automatically select the grub install device. It works
fine if the disk is always the same, e.g., /dev/sda, but does
not work well when using the same preseed file on systems withi
different types of disks, e.g., /dev/vda and /dev/hda.

Setting grub-installer/bootdev to "default" will automatically
select the first disk for grug installation. It works as long
as there is only one choice of disk to install on and thus
avoids the "Please select /dev/... or none" question.

diff --git a/grub-installer b/grub-installer
index 50155a9..12f9a31 100755
--- a/grub-installer
+++ b/grub-installer
@@ -678,7 +678,11 @@ while : ; do
log "q='$q' state='$state' defbd='$default_bootdev' bd='$bootdev'"
db_fget grub-installer/bootdev seen
if [ "$RET" = true ] && db_get grub-installer/bootdev && [ "$RET" ] ; 
then
-   bootdev=$RET
+   if  [ "$RET" = "default" ]; then
+   bootdev=$default_bootdev
+   else
+   bootdev=$RET
+   fi
break
elif [ "$state" = 1 ]; then
db_input high $q || true


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org



Bug#455311: Fixed in spew-1.0.7

2008-05-13 Thread Andrew Patterson
I have a fix for this in spew-1.0.7.  Which can be found at:

ftp://ftp.berlios.de/pub/spew/1.0.7/index.html

I had to add  includes to handle the normal strlen, memcpy,
stuff.

Tested with gcc-4.3.

-- 
Andrew Patterson




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]



Bug#334305: Found off-by-one error in ioc_conv

2005-11-09 Thread Andrew Patterson
The following patch fixes an off-by-one error in ioc_conv().  Calls to
this function are corrupting device names (at least on IA-64), so
comparing the device name to the name in diskstats always fails.  Hence
no disk output.

This patch will apply to both the upstream tarball and the debian
package.

Andrew


Index: sysstat-6.0.1/ioconf.c
===
--- sysstat-6.0.1.orig/ioconf.c 2005-06-25 03:02:47.0 -0600
+++ sysstat-6.0.1/ioconf.c  2005-11-09 17:50:58.470114529 -0700
@@ -89,7 +89,7 @@ static void ioc_free(void)
 static char *ioc_conv(int radix, int nozero, const char *syms,
  unsigned int val)
 {
-   static char out[16];
+   static char out[17];
char *p;
int j;
 




-- 
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]