[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
All the things sound great here, imho we should do all of them. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Branch linked: lp:livecd-rootfs -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Changed in: livecd-rootfs (Ubuntu Cosmic) Status: New => Fix Committed ** Changed in: livecd-rootfs (Ubuntu Bionic) Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
smoser, The cloud-initramfs-copymods package is in the server seed and it depends on /lib/modules existing for RO root FS. So it can own the directory and we'll fix this in the least magic way possible. https://code.launchpad.net/~rcj/cloud-initramfs-tools/+git/cloud- initramfs-tools/+merge/355409 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Merge proposal linked: https://code.launchpad.net/~rcj/cloud-initramfs-tools/+git/cloud-initramfs-tools/+merge/355409 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Also affects: cloud-initramfs-tools (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
Hi, Can you please add a gating check on image publication that /lib/modules directory exists? ** Changed in: open-iscsi (Ubuntu Xenial) Importance: Undecided => Low ** Changed in: open-iscsi (Ubuntu Xenial) Status: New => Confirmed ** Changed in: open-iscsi (Ubuntu Bionic) Importance: Undecided => Low ** Changed in: open-iscsi (Ubuntu Bionic) Status: New => Confirmed ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel - modules are copied to a tempfs or other overlay mounted on /lib/modules. + modules are copied from the initramfs to the real root via cloud- + initramfs-copymods package. * This affects users of our stable release images available from http ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need ubuntu-old-fashioned master for cosmic) * Re-build the images using the updated livecd-rootfs package. * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using unsquashfs again. * Inspect the unpacked root filesystem and find that '/lib/modules' exists. * It is pure luck that package purges which are done analogously in Cosmic image builds do not remove '/lib/modules', hence this fix is introduced there, as well. * Xenial is not affected. * Test builds were carried out for Cosmic and Bionic with the expected results. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- factoring. See also: https://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/bionic- proposed/revision/1678 https://bazaar.launchpad.net/~ubuntu-core-dev/livecd- rootfs/trunk/revision/1681 * Packaging tools should not take offense at the existence of a directory, even if it was not part of a package. So potential for unforseeable regressions is very low. ===ORIGINAL BUG DESCRIPTION=== Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images available from http ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need ubuntu-old-fashioned master for cosmic) * Re-build the images using the updated livecd-rootfs package. * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using unsquashfs again. * Inspect the unpacked root filesystem and find that '/lib/modules' exists. * It is pure luck that package purges which are done analogously in Cosmic image builds do not remove '/lib/modules', hence this fix is introduced there, as well. * Xenial is not affected. * Test builds were carried out for Cosmic and Bionic with the expected results. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- - factoring. + factoring. See also: + + https://bazaar.launchpad.net/~ubuntu-core-dev/livecd-rootfs/bionic- + proposed/revision/1678 + + https://bazaar.launchpad.net/~ubuntu-core-dev/livecd- + rootfs/trunk/revision/1681 * Packaging tools should not take offense at the existence of a directory, even if it was not part of a package. So potential for unforseeable regressions is very low. ===ORIGINAL BUG DESCRIPTION=== Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
The attachment "debdiff for Cosmic." seems to be a debdiff. The ubuntu- sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team. [This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.] ** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
debdiff for Cosmic. ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images available from http ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need ubuntu-old-fashioned master for cosmic) * Re-build the images using the updated livecd-rootfs package. * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using unsquashfs again. * Inspect the unpacked root filesystem and find that '/lib/modules' exists. - * Do the above for Bionic and Cosmic. + * It is pure luck that package purges which are done analogously in + Cosmic image builds do not remove '/lib/modules', hence this fix is + introduced there, as well. + + * Xenial is not affected. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- factoring. * Packaging tools should not take offense at the existence of a directory, even if it was not part of a package. So potential for unforseeable regressions is very low. === Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images available from http ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need ubuntu-old-fashioned master for cosmic) * Re-build the images using the updated livecd-rootfs package. * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using unsquashfs again. * Inspect the unpacked root filesystem and find that '/lib/modules' exists. * It is pure luck that package purges which are done analogously in Cosmic image builds do not remove '/lib/modules', hence this fix is introduced there, as well. * Xenial is not affected. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- factoring. * Packaging tools should not take offense at the existence of a directory, even if it was not part of a package. So potential for unforseeable regressions is very low. - === + ===ORIGINAL BUG DESCRIPTION=== Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images available from http ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
debdiff for Bionic. ** Patch added: "debdiff for Bionic." https://bugs.launchpad.net/cloud-images/+bug/1792905/+attachment/5190910/+files/livecd-rootfs-bionic.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] - * TODO... + * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- + cloudimg-amd64.squashfs - * these should allow someone who is not familiar with the affected - package to reproduce the bug and verify that the updated package fixes - the problem. + * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` + + * Inspect the unpacked root filesystem and find that '/lib/modules' is + missing. + + * Install local build scripts as described at + https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need + ubuntu-old-fashioned master for cosmic) + + * Re-build the images using the updated livecd-rootfs package. + + * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using + unsquashfs again. + + * Inspect the unpacked root filesystem and find that '/lib/modules' + exists. + + * Do the above for Bionic and Cosmic. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- factoring. * Packaging tools should not take offense at the existence of a directory, even if it was not part of a package at that time. So potential for regressions from this fix is basically zero. === Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. * This affects users of our stable release images. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs - * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` + * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` - * Inspect the unpacked root filesystem and find that '/lib/modules' is + * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at https://github.com/chrisglass/ubuntu-old-fashioned (note: you will need ubuntu-old-fashioned master for cosmic) * Re-build the images using the updated livecd-rootfs package. * Unpack the resulting livecd.ubuntu-cpc.squashfs artifact using unsquashfs again. * Inspect the unpacked root filesystem and find that '/lib/modules' exists. * Do the above for Bionic and Cosmic. [Regression Potential] * This is a fix to a regression. The existence of the directory had previously been ensured, but the mkdir call got lost in recent re- factoring. * Packaging tools should not take offense at the existence of a - directory, even if it was not part of a package at that time. So - potential for regressions from this fix is basically zero. + directory, even if it was not part of a package. So potential for + unforseeable regressions is very low. === Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. ** Description changed: [Impact] * Affects environments where the base image is read-only but kernel modules are copied to a tempfs or other overlay mounted on /lib/modules. - * This affects users of our stable release images. + * This affects users of our stable release images available from http + ://cloud-images.ubuntu.com. * The attached fixes ensure /lib/modules always exists by creating it explicitly instead of relying on it to come from a package. [Test Case] * Download http://cloud-images.ubuntu.com/bionic/current/bionic-server- cloudimg-amd64.squashfs * Unpack it via `sudo unsquashfs bionic-server-cloudimg-amd64.squashfs` * Inspect the unpacked root filesystem and find that '/lib/modules' is missing. * Install local build scripts as described at
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Description changed: - #! NB Foundations coding day task !# - [Impact] - * An explanation of the effects of the bug on users and + * Affects environments where the base image is read-only but kernel + modules are copied to a tempfs or other overlay mounted on /lib/modules. - * justification for backporting the fix to the stable release. + * This affects users of our stable release images. - * In addition, it is helpful, but not required, to include an -explanation of how the upload fixes this bug. + * The attached fixes ensure /lib/modules always exists by creating it + explicitly instead of relying on it to come from a package. [Test Case] - * detailed instructions how to reproduce the bug + * TODO... - * these should allow someone who is not familiar with the affected -package to reproduce the bug and verify that the updated package fixes -the problem. + * these should allow someone who is not familiar with the affected + package to reproduce the bug and verify that the updated package fixes + the problem. [Regression Potential] - * discussion of how regressions are most likely to manifest as a result - of this change. + * This is a fix to a regression. The existence of the directory had + previously been ensured, but the mkdir call got lost in recent re- + factoring. - * It is assumed that any SRU candidate patch is well-tested before -upload and has a low overall risk of regression, but it's important -to make the effort to think about what ''could'' happen in the -event of a regression. - - * This both shows the SRU team that the risks have been considered, -and provides guidance to testers in regression-testing the SRU. - - [Other Info] - - * Anything else you think is useful to include - * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board - * and address these questions in advance + * Packaging tools should not take offense at the existence of a + directory, even if it was not part of a package at that time. So + potential for regressions from this fix is basically zero. === Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
Change not needed for xenial, livecd-rootfs is already creating '/lib/modules' on the rootfs tarball. ** Changed in: livecd-rootfs (Ubuntu Xenial) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Also affects: livecd-rootfs (Ubuntu) Importance: Undecided Status: New ** Also affects: open-iscsi (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: livecd-rootfs (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: open-iscsi (Ubuntu Cosmic) Importance: Low Status: Confirmed ** Also affects: livecd-rootfs (Ubuntu Cosmic) Importance: Undecided Status: New ** Also affects: open-iscsi (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: livecd-rootfs (Ubuntu Xenial) Importance: Undecided Status: New ** Description changed: + #! NB Foundations coding day task !# + + [Impact] + + * An explanation of the effects of the bug on users and + + * justification for backporting the fix to the stable release. + + * In addition, it is helpful, but not required, to include an +explanation of how the upload fixes this bug. + + [Test Case] + + * detailed instructions how to reproduce the bug + + * these should allow someone who is not familiar with the affected +package to reproduce the bug and verify that the updated package fixes +the problem. + + [Regression Potential] + + * discussion of how regressions are most likely to manifest as a result + of this change. + + * It is assumed that any SRU candidate patch is well-tested before +upload and has a low overall risk of regression, but it's important +to make the effort to think about what ''could'' happen in the +event of a regression. + + * This both shows the SRU team that the risks have been considered, +and provides guidance to testers in regression-testing the SRU. + + [Other Info] + + * Anything else you think is useful to include + * Anticipate questions from users, SRU, +1 maintenance, security teams and the Technical Board + * and address these questions in advance + + === + Let me first start with saying MAAS is *not* using iSCSI anymore and is *NOT* in this case either. For some reason now using enlistment, commissioning, and deploying the ephemeral environment will block for 1 min 30 seconds waiting for the iSCSI daemon to succeed, which it never does. This increases the boot time drastically. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
I put a pull request to upstream open-iscsi to improve its failure path here : https://github.com/open-iscsi/open-iscsi/pull/127 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
Urgency is critical. Customer deployments that use modules will be broken. This would include vfat, zfs... many things. I believe this is the root cause of Christian's maas deployment with console log at http://paste.ubuntu.com/p/BMS4dbW4XD/ ** Changed in: cloud-images Importance: High => Critical -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
Urgency is high, this slows down every booting machine of MAAS by 1:30 seconds, that is a long boot time just to get the OS to install or to commission the machine. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
It appears that the reason for lack of `/lib/modules` is the result of removal of the kernel image and modules happening in a slightly different order then typical. In the case of 20180911, the kernel image was removed first, then the kernel modules. This resulted in complete clean-up of `/lib/modules`. This order appears to be arbitrary and we just got lucky (or unlucky) this time. We can make image build changes to ensure that `/lib/modules` is present in the bionic and xenial squashfs. However, is there a different solution we should be pursuing for cosmic that doesn't require this directory to be present? Finally, what is the urgency of getting this issue resolved. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
just for reference, here is iscsid status when it fails. the error is 'can not create NETLINK_ISCSI socket'. I'm confused as to why, but it will start later in boot fine. $ systemctl status iscsid.service --no-pager --full ● iscsid.service - iSCSI initiator daemon (iscsid) Loaded: loaded (/lib/systemd/system/iscsid.service; enabled; vendor preset: enabled) Active: failed (Result: timeout) since Wed 2018-09-19 10:20:28 UTC; 2min 38s ago Docs: man:iscsid(8) Sep 19 10:18:57 ubuntu systemd[1]: Starting iSCSI initiator daemon (iscsid)... Sep 19 10:18:57 ubuntu iscsid[758]: iSCSI logger with pid=765 started! Sep 19 10:18:57 ubuntu systemd[1]: iscsid.service: Failed to parse PID from file /run/iscsid.pid: Invalid argument Sep 19 10:18:57 ubuntu iscsid[765]: iSCSI daemon with pid=767 started! Sep 19 10:18:57 ubuntu iscsid[765]: can not create NETLINK_ISCSI socket Sep 19 10:20:28 ubuntu systemd[1]: iscsid.service: Start operation timed out. Terminating. Sep 19 10:20:28 ubuntu systemd[1]: iscsid.service: Failed with result 'timeout'. Sep 19 10:20:28 ubuntu systemd[1]: Failed to start iSCSI initiator daemon (iscsid). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
I want to be clear and state that there will be other fallout from not having any modules in the ephemeral environment. Just fixing open-iscsi to not run or not fail doesn't solve the problem. Also note that if you boot the ephemeral environment without 'ro' on the command line, then this particular case will not be a problem, but that isn't really a solution either. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
See bug 1543204 for more information. The change that caused this regression was in ubuntu released bionic images. Between 20180831 and 20180911 the /lib/modules directory disappeared. That means that 'copymods' cannot copy modules from initramfs into the root filesystem, and without that, open-iscsi is behaving oddly. We can/should make open-iscsi not block in this case as whatever it is waiting for is probably not going to ever arrive (possibly a kernel module?). But we want/need the /lib/modules directory in the images or copymods can't really do what it does. ** Changed in: open-iscsi (Ubuntu) Status: New => Confirmed ** Changed in: open-iscsi (Ubuntu) Importance: Undecided => Low ** Also affects: cloud-images Importance: Undecided Status: New ** Changed in: cloud-images Status: New => Triaged ** Changed in: cloud-images Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
Here is the output log: http://paste.ubuntu.com/p/KfTWC5ghwR/ You can't really see the iSCSI error in the console log. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1792905] Re: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds
** Also affects: open-iscsi Importance: Undecided Status: New ** No longer affects: open-iscsi ** Also affects: open-iscsi (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1792905 Title: [2.5] iSCSI systemd services fails and blocks for 1 min 30 secconds To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1792905/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
