On Mon, 20 Jun 2016 16:40:29 -0600 Andrew Patterson <andrew.patter...@hpe.com>
wrote:
> On Thu, 16 Jun 2016 15:19:22 +0530 Ritesh Raj Sarraf <r...@debian.org> wrote:
> > Control: tag -1 +pending
> >
> > On Wed, 2016-06-15 at 13:03 -0600, Andrew Patterson wrote:
> > > For most users this is not an issue. How many systems are going to use
> > > that many LUNs?
> > >
> >
> > I agree.
> >
> > > One counter-example is using one or more FC LUNs per VM guest on a
> > > host. In such a case, one can just modify /etc/sysctl.conf to increase
> > > the setting.
> >
> > Yes. Most hypervisors may want this. But again, like you said, most users
> > do not
> > map that many LUNs. And recommended practice is to have a VM backed by a
> > file
> > backend mostly.
>
>
> OpenStack/cinder can (and is) be uses to provision VMs with networked
> drives. It is pretty common for cloud applications. I don't know what
> a practical limit for the number of volume per VMs per host should be.
>
>
> >
> > > We can run into problems when using
> > > multipath-tools-boot. In this case, we use the default kernel settings
> > > for aio-max-nr since multipath is run from the initramfs. I have
> > > worked around this by modifying the script in
> > > /usr/share/initramfs/scripts/local-top/multipath to temporarilly set
> > > aio-max-nr to 1048576 while doing discovery.
> >
> >
> > So, I'll just add this to README.Debian. There's not much that we can do
> > beyond
> > documenting such behavior.
>
> I will work on patch to have a user-configurable setting of aio-max-nr
> in the initramfs. You can then decide if it is worth it to apply it.
>
> In the meantime, the README.Debian change sounds fine to me.
>
Here is a patch that copies the fs.aio-max-nr setting to the
initramfs. There is more to do to solve the problem with booting with
many LUNs, but this is a prerequisite for completely solving that
issue. This patch assumes that the patch in:
https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=827407;filename=multipath-tools-827407.patch;msg=30
has been applied. I can rework it if that one is not accepted.
Andrew
--
Andrew Patterson
Hewlett-Packard Enterprise
commit e68651ca8b6a9c2b8afee787b3e5221b402846fd
Author: Andrew Patterson <andrew.patter...@hpe.com>
Date: Thu Jun 23 22:34:16 2016 +0000
Use fs.aio-max-nr setting in initramfs
We sometimes need to increase the fs.aio-max-nr from the kernel default
value when running multipath discovery in the initramfs with large numbers
of LUNs. This patch copies the setting there and temporarily changes it
while performing discovery.
diff --git a/debian/initramfs/hooks b/debian/initramfs/hooks
index 94cb7e8..a29dbac 100644
--- a/debian/initramfs/hooks
+++ b/debian/initramfs/hooks
@@ -63,4 +63,4 @@ for x in $MULTIPATH_TOOLS_MODULES; do
done
echo "MULTIPATH_TOOLS_HW_HANDLERS=\"$MULTIPATH_TOOLS_HW_HANDLERS\"" > $DESTDIR/etc/multipath-tools-boot.conf
echo "MULTIPATH_TOOLS_MODULES=\"$MULTIPATH_TOOLS_MODULES\"" >> $DESTDIR/etc/multipath-tools-boot.conf
-
+echo "MULTIPATH_TOOLS_MIN_AIO_MAX_NR=$(sysctl fs.aio-max-nr | cut -f3 -d ' ')" >> $DESTDIR/etc/multipath-tools-boot.conf
diff --git a/debian/initramfs/local-top b/debian/initramfs/local-top
index 99e4222..67698bd 100644
--- a/debian/initramfs/local-top
+++ b/debian/initramfs/local-top
@@ -31,10 +31,14 @@ if [ ! -e /sbin/multipath ]; then
exit 0
fi
+# Save boot setting for aio-max-nr.
+BOOT_AIO_MAX_NR=$(cat /proc/sys/fs/aio-max-nr)
+
if [ -f /etc/multipath-tools-boot.conf ]; then
. /etc/multipath-tools-boot.conf
else
MULTIPATH_TOOLS_MODULES="dm-mod dm-multipath"
+ MULTIPATH_TOOLS_MIN_AIO_MAX_NR=$BOOT_AIO_MAX_NR
fi
verbose && log_begin_msg "Loading multipath modules"
@@ -47,6 +51,12 @@ for module in ${MULTIPATH_TOOLS_MODULES}; do
done
verbose && log_end_msg
+
+# Adjust aio-max-nr if too low.
+if [ $BOOT_AIO_MAX_NR -le $MULTIPATH_TOOLS_MIN_AIO_MAX_NR ]; then
+ echo "$MULTIPATH_TOOLS_MIN_AIO_MAX_NR" > /proc/sys/fs/aio-max-nr
+fi
+
verbose && log_begin_msg "Discovering multipaths"
/sbin/multipath -v $VERBOSITY
verbose && log_end_msg
@@ -59,6 +69,9 @@ if [ -x /sbin/udevadm ]; then
/sbin/udevadm settle --timeout=10
fi
+# Restore aio-max-nr.
+echo "$BOOT_AIO_MAX_NR" > /proc/sys/fs/aio-max-nr
+
maybe_break post-multipath
exit 0
diff --git a/debian/multipath-tools.README.Debian b/debian/multipath-tools.README.Debian
index c1f50a2..2665046 100644
--- a/debian/multipath-tools.README.Debian
+++ b/debian/multipath-tools.README.Debian
@@ -35,7 +35,8 @@ To make this setting persistent a line may be added to /etc/sysctl.conf:
Consult appropriate application and operating system tuning recommendations for
guidance on appropriate values for this parameter.
-
+If using multipath-tools-boot, remember to run update-initramfs so the
+adjusted aio-max-nr will also be set in the initramfs.
What are these weird numbers in /dev/mapper?
