Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
* dann frazier <1918...@bugs.launchpad.net> [2021-03-19 12:16]:
> On Fri, Mar 19, 2021 at 10:01 AM Ryan Harper <1918...@bugs.launchpad.net>
> wrote:
> >
> > * dann frazier <1918...@bugs.launchpad.net> [2021-03-18 16:30]:
> > > On Thu, Mar 18, 2021 at 12:25 PM Ryan Harper <1918...@bugs.launchpad.net>
> > > wrote:
> > > >
> > > > * dann frazier <1918...@bugs.launchpad.net> [2021-03-18 12:11]:
> > > > > On Thu, Mar 18, 2021 at 10:25 AM Ryan Harper
> > > > > <1918...@bugs.launchpad.net> wrote:
> > > > > >
> > > > > > * dann frazier <1918...@bugs.launchpad.net> [2021-03-17 20:30]:
> > > > > > > On Tue, Mar 16, 2021 at 10:05 AM Ryan Harper
> > > > > > > <1918...@bugs.launchpad.net> wrote:
> > > > > > > >
> > > > > > > > Hi Dan,
> > > > > > > >
> > > > > > >
> > > > > > > 1) flash-kernel could get installed post-divert. In that case,
> > > > > > > flash-kernel's own postinst will cause it to run and then fail.
> > > > > > > This
> > > > > > > happens today if you start with a cloud image w/o flash-kernel
> > > > > > > pre-baked because Ubuntu's kernel recommends flash-kernel,
> > > > > > > causing it
> > > > > > > to be installed along with the kernel. Official cloud images
> > > > > > > happen to
> > > > > >
> > > > > > Hrm, so if we take a squashfs rootfs (with no flash-kernel present)
> > > > > > chroot into it and install the linux-image-generic package pulling
> > > > > > in
> > > > > > flash-kernel this fails due to postinst of flash-kernel expecting
> > > > > > initramfs to already be generated? This doesn't seem like a curtin
> > > > > > bug.
> > > > >
> > > > > If done so in a chroot that exposes the kernel interfaces (/proc &
> > > > > /sys) that claim to be hardware that requires the initramfs to be
> > > > > post-processed, yes.
> > > >
> > > > Maybe I'm missing something but if I install linux-image-generic
> > > > it populates /boot with vmlinuz-$version (and a few more things)
> > > > and /lib/modules/$version and the kernels postinst will invoke
> > > > update-initramfs. The /boot/initrd.img-$version is *generated* at
> > > > that time during the kernel's postinstall
> > > >
> > > > Now, in the arm case IIUC, the kernel package has a dep on flash-kernel
> > > > being present as it's "needed" to generate the initramfs ... so how can
> > > > flash-kernel's postinst *fail* if it is the tool that's generating said
> > > > initramfs file?
> > >
> > > What flash-kernel does is generate wrapped versions of *exisiting*
> > > vmlinuz and initrd.img files. It doesn't generate those files, rather
> > > post-processes them.
> > > The kernel doesn't depend on flash-kernel, it just recommends it like
> > > it does GRUB on x86.
> >
> > Yes, I get that but it still looks like a packaging bug if dpkg installs
> > flash-kernel first and /boot is not populated with existing initrds; one
> > could easily see this happen in a debootstrap.
>
> Given that a failure to produce a wrapped initrd could cause a system
> to become unbootable, it does seem to me like a hard failure here is
> warranted. But, perhaps we could provide a "s... it's ok, just
> chill" mechanism. Maybe a FLASH_KERNEL_SKIP=1 environment variable?
Agreed but with the condition that the *input* is present. If the
initrd file has not yet been generated then another error from
flash-kernel seems redundant, specifically in the case where if the
kernel package is not yet installed (and maybe this is the reasonble
check the post-inst can do) then it's *always* going to fail.
Does flash-kernel hook into initramfs updates via the
/etc/kernel/{pre,post}inst.d/flash-kernel ? like initramfs-tools does?
I suspect most of what flash-kernel does would be triggered via kernel
postinst hooks.
>
> > Is the "liveness" of the chroot what's tripping up flash-kernel? We
> > currently run inside a chroot which mounts /dev /proc /run and /sys; we
> > could drop those but it also seems reasonable to have flash-kernel not
> > expect existing initrds?
>
> Certainly a non-live chroot can avoid this by leading f-k to believe
> it does not recognize the system. In fact, ISTR bind mounting certain
> files in build chroots to trick f-k into doing nothing.
Interesting. That may be another approach; though that would be a f-k
specific mount for curtin to swallow when installing one package; we
might not know it's getting installed (like as a Recommends with the
linux-image-generic) so that might not be as useful.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1918427
Title:
curtin: install flash-kernel in arm64 UEFI unexpected
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
* dann frazier <1918...@bugs.launchpad.net> [2021-03-18 16:30]: > On Thu, Mar 18, 2021 at 12:25 PM Ryan Harper <1918...@bugs.launchpad.net> > wrote: > > > > * dann frazier <1918...@bugs.launchpad.net> [2021-03-18 12:11]: > > > On Thu, Mar 18, 2021 at 10:25 AM Ryan Harper <1918...@bugs.launchpad.net> > > > wrote: > > > > > > > > * dann frazier <1918...@bugs.launchpad.net> [2021-03-17 20:30]: > > > > > On Tue, Mar 16, 2021 at 10:05 AM Ryan Harper > > > > > <1918...@bugs.launchpad.net> wrote: > > > > > > > > > > > > Hi Dan, > > > > > > > > > > > > > > > > 1) flash-kernel could get installed post-divert. In that case, > > > > > flash-kernel's own postinst will cause it to run and then fail. This > > > > > happens today if you start with a cloud image w/o flash-kernel > > > > > pre-baked because Ubuntu's kernel recommends flash-kernel, causing it > > > > > to be installed along with the kernel. Official cloud images happen to > > > > > > > > Hrm, so if we take a squashfs rootfs (with no flash-kernel present) > > > > chroot into it and install the linux-image-generic package pulling in > > > > flash-kernel this fails due to postinst of flash-kernel expecting > > > > initramfs to already be generated? This doesn't seem like a curtin bug. > > > > > > If done so in a chroot that exposes the kernel interfaces (/proc & > > > /sys) that claim to be hardware that requires the initramfs to be > > > post-processed, yes. > > > > Maybe I'm missing something but if I install linux-image-generic > > it populates /boot with vmlinuz-$version (and a few more things) > > and /lib/modules/$version and the kernels postinst will invoke > > update-initramfs. The /boot/initrd.img-$version is *generated* at > > that time during the kernel's postinstall > > > > Now, in the arm case IIUC, the kernel package has a dep on flash-kernel > > being present as it's "needed" to generate the initramfs ... so how can > > flash-kernel's postinst *fail* if it is the tool that's generating said > > initramfs file? > > What flash-kernel does is generate wrapped versions of *exisiting* > vmlinuz and initrd.img files. It doesn't generate those files, rather > post-processes them. > The kernel doesn't depend on flash-kernel, it just recommends it like > it does GRUB on x86. Yes, I get that but it still looks like a packaging bug if dpkg installs flash-kernel first and /boot is not populated with existing initrds; one could easily see this happen in a debootstrap. Is the "liveness" of the chroot what's tripping up flash-kernel? We currently run inside a chroot which mounts /dev /proc /run and /sys; we could drop those but it also seems reasonable to have flash-kernel not expect existing initrds? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
* dann frazier <1918...@bugs.launchpad.net> [2021-03-18 12:11]: > On Thu, Mar 18, 2021 at 10:25 AM Ryan Harper <1918...@bugs.launchpad.net> > wrote: > > > > * dann frazier <1918...@bugs.launchpad.net> [2021-03-17 20:30]: > > > On Tue, Mar 16, 2021 at 10:05 AM Ryan Harper <1918...@bugs.launchpad.net> > > > wrote: > > > > > > > > Hi Dan, > > > > > > > > > > 1) flash-kernel could get installed post-divert. In that case, > > > flash-kernel's own postinst will cause it to run and then fail. This > > > happens today if you start with a cloud image w/o flash-kernel > > > pre-baked because Ubuntu's kernel recommends flash-kernel, causing it > > > to be installed along with the kernel. Official cloud images happen to > > > > Hrm, so if we take a squashfs rootfs (with no flash-kernel present) > > chroot into it and install the linux-image-generic package pulling in > > flash-kernel this fails due to postinst of flash-kernel expecting > > initramfs to already be generated? This doesn't seem like a curtin bug. > > If done so in a chroot that exposes the kernel interfaces (/proc & > /sys) that claim to be hardware that requires the initramfs to be > post-processed, yes. Maybe I'm missing something but if I install linux-image-generic it populates /boot with vmlinuz-$version (and a few more things) and /lib/modules/$version and the kernels postinst will invoke update-initramfs. The /boot/initrd.img-$version is *generated* at that time during the kernel's postinstall Now, in the arm case IIUC, the kernel package has a dep on flash-kernel being present as it's "needed" to generate the initramfs ... so how can flash-kernel's postinst *fail* if it is the tool that's generating said initramfs file? > > In summary curtin will need: > > > > move ephemeral deps.py flash-kernel to arch-packages in > > install-missing-packages with the same logic guarding when to add the > > dep. > > > > It's not clear to me why curtin should work around the packaging bugs > > around flash-kernel and I would suggest that flash-kernel be kept in the > > cloud images until the packging deps/bugs around it are fixed. > > I don't think it should - we should SRU Date's f-k change and the > kernel Recommends change. Are there other packaging issues you've > identified? I don't think possibly something flash-kernel related to the above discussion but that may just be my own misinformation. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
* dann frazier <1918...@bugs.launchpad.net> [2021-03-17 20:30]: > On Tue, Mar 16, 2021 at 10:05 AM Ryan Harper <1918...@bugs.launchpad.net> > wrote: > > > > Hi Dan, > > > > Could you summarize the problem with flash-kernel and this system? > > Sure. flash-kernel recognizes Mustang boards and will generate uImage > and uInitrd files for it, which are required for booting with u-boot > firmware. However, these boards can also run in UEFI mode, which > Date's board does. In UEFI mode, flash-kernel still knows it is on a > Mustang and generates uImage/uInitrd files - which won't be used for > anything in that case, they are just wasting space, but does not cause > it to fail. This does cause problems in a curtin install though. > Curtin has logic to divert away tools that get executed during > initramfs hooks, to avoid failures in packaging scripts before an > initramfs is generated. flash-kernel in particular will fail if an > initramfs is not found on this system. Curtin tries to be smart here > and only divert flash-kernel 1) if it is installed and 2) on systems > that are*not* in UEFI mode, and both of these scenarios have escapes: > > 1) flash-kernel could get installed post-divert. In that case, > flash-kernel's own postinst will cause it to run and then fail. This > happens today if you start with a cloud image w/o flash-kernel > pre-baked because Ubuntu's kernel recommends flash-kernel, causing it > to be installed along with the kernel. Official cloud images happen to Hrm, so if we take a squashfs rootfs (with no flash-kernel present) chroot into it and install the linux-image-generic package pulling in flash-kernel this fails due to postinst of flash-kernel expecting initramfs to already be generated? This doesn't seem like a curtin bug. > have flash-kernel pre-baked which avoids this issue. I think curtin > should work whether or not the kernel recommends flash-kernel and > whether or not curtin is pre-baked (in fact, I'd like for us to stop > pre-baking it - the vast majoriy of ARM servers do not need it). > > 2) If flash-kernel is installed, and curtin finds we're in UEFI mode, > it chooses not to divert flash-kernel. flash-kernel will therefore run > and fail on UEFI Mustangs. I don't think that's true. The logic for disabling initramfs tools always runs regardless of UEFI mode and arch. See curtin/commands/curthooks.py:builtin_curthooks() lines 1692- 1699 > > The way I've personally framed this issue is that Ubuntu should not be > trying to install flash-kernel on ARM systems that don't require it, > which is the reason I've added the various tasks here. > - cloud images shouldn't prebake it OK > - the kernel should allow non-flash-kernel bootloaders to satisfy its > recommends OK Thanks. I replied to your later post without seeing this first. This helps a lot. > - curtin shouldn't install flash-kernel on efi-based arm64 servers. > It does this today, but - in what seems like a bug, only in the > ephemeral and not the target. Yeah; I suspect flash-kernel being pre-baked into images hid this for some time. As I mentioned in the other reply, I do think that lines 57-58 in curtin/deps/__init__.py which bring in flash-kerenl to the epheramal can be removed. And it sounds like we'd move that logic instead to the install-missing-packages arch_packages where we ensure s390-tools/zipl are install for s390, there we'd add the same logic to append flash-kernel where needed for the target OS. > > A separate issue is that flash-kernel should know to just exit if it's > running on an EFI system and not bother creating the unused > uImage/uInitrd - Date recently got a patched merged into Debian's f-k > to do that. That would seemingly also avoid the curtin issues here, > but only if we continue to install flash-kernel all the time. OK. In summary curtin will need: move ephemeral deps.py flash-kernel to arch-packages in install-missing-packages with the same logic guarding when to add the dep. It's not clear to me why curtin should work around the packaging bugs around flash-kernel and I would suggest that flash-kernel be kept in the cloud images until the packging deps/bugs around it are fixed. Does that make sense? Ryan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
* dann frazier <1918...@bugs.launchpad.net> [2021-03-17 20:40]: > On Wed, Mar 17, 2021 at 4:56 PM Ryan Harper <1918...@bugs.launchpad.net> > wrote: > > > > I still don't understand: > > > > 1) why does which not find flash-kernel if it's present in the ephemeral > > image (meaning it will also be present in the target filesystem > > 2) What is the problem with flash-kernel such that you need to dpkg-divert > > it? > > > > Generally, we do not want to include paths to binaries; we install > > Ubuntu and Centos, and possibly other distros which may not have the > > same path to this tools so I would like to understand what's not working > > to understand why we're hardcoding paths to binaries for dpkg-divert. > > In my POC patch, it is because we need to divert flash-kernel before > it is installed, *just in case* it gets installed as a dependency of Is flash-kernel supposed to be in the cloud-image or not? Is flash-kernel supposed to be in the target OS or not? > some other package (in our case it happened due to a kernel Recommends > relationship). We therefore can't use `which` to look up the path... > unless we continue to always install f-k in the ephemeral for the sole > purpose of looking up its path... but that seems to making assumptions > as well - i.e. that the ephemeral OS has the same paths as the target > OS. flash-kernel runs in the target OS; if it's installed in ephemeral environment thats a by-product of obtaining tools needed but not present in the ephemeral environment. curtin/deps.py runs to ensure the ephemeral environment has the needed tools to create whatever storage configuration curtin supports. In the squashfs images which do not have a kernel already installed curtin will install linux-image-generic to make *additional* kernel modules which may not be in the initrd that maas produces available. I'm not sure why the maas initrd doesn't yet have zfs kernel modules but that's typically the driver of the linux-image-generic install. w.r.t the dep on flash-kernel, if that is only supposed to run in the target image, then I agree we can remove lines 57-58 in curtin/dep/__init.py and not install the package (though it may still get pulled in via the kernel package install). Later durting curthooks stage, curtin will query the *target* os in install-missing-packages for expected packages needed in the target OS. Prior to installing packages in the target OS, curtin does attempt to divert initramfs triggering tools to prevent generating the initramfs more than once (this happens when we install a kernel package in the target OS and either updates or other packages trigger initramfs rebuilds). This is designed around the assumption that the initramfs generation tool is present in the image but not tied to the kernel pacakge. For Ubuntu this is initramfs-tools along with some arch specific tools. flash-kernel is the tool used on arm for non-uefi boot. I see a couple of options but I think I'd like to understand: 1) when is flash-kernel required? Curtin current expects flash-kernel to be needed when arch is arm and when booting non-uefi. Is this still accurate? 2) Is flash-kernel always installed in arm cloud images used by maas? 3) Is flash-kernel installed by packages due to Recommends when it's not needed? If so, are there packaging bugs that need fixing? Ryan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
I still don't understand: 1) why does which not find flash-kernel if it's present in the ephemeral image (meaning it will also be present in the target filesystem 2) What is the problem with flash-kernel such that you need to dpkg-divert it? Generally, we do not want to include paths to binaries; we install Ubuntu and Centos, and possibly other distros which may not have the same path to this tools so I would like to understand what's not working to understand why we're hardcoding paths to binaries for dpkg-divert. Thanks, Ryan Ryan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1916325] Re: Overly strict NAME_REGEX rejects valid hostnames
Hi, Thank you for filing a bug and helping make Ubuntu better. I believe the regex restriction is happening in the Ubuntu Server Live Installer (subiquity). I've added that package to this bug. Here's the source code in question you're hitting. https://github.com/canonical/subiquity/blob/a76581cd2b973b55e55c6ac05b5bf47168493140/subiquity/ui/views/identity.py#L110 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1916325 Title: Overly strict NAME_REGEX rejects valid hostnames To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1916325/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1916325] Re: Overly strict NAME_REGEX rejects valid hostnames
** Also affects: curtin Importance: Undecided Status: New ** Also affects: subiquity (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1916325 Title: Overly strict NAME_REGEX rejects valid hostnames To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1916325/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1918427] Re: curtin: install flash-kernel in arm64 UEFI unexpected
Hi Dan, Could you summarize the problem with flash-kernel and this system? * dann frazier <1918...@bugs.launchpad.net> [2021-03-15 18:25]: > Attached is a patch for curtin that works for me, though it could use > some cleanup. It installs flash-kernel in the same place GRUB gets > installed for EFI-based systems, which seems like the appropriate place. > Before installint it, it diverts /usr/sbin/flash-kernel so that flash- > kernel's postinst doesn't try (and fail) to flash a kernel that isn't > yet installed. There's existing code in enable_update_initramfs() that > will undo this diversion. I suspect we should refactor > disable_update_initramfs() somehow instead of duplicating this code like > I've done here. Looking at the patch, it looks like you could instead modify _update_initramfs_tools() to only return flash-kernel in the case that it's needed rather than for all aarch64. Alternatively, enable/disable_update_initramfs could take a force boolean which would perform the operation whether the tool is present or not (though I don't know if dpkg-divert cares about the presence of the binary-to-be-diverted). > > I've tested this w/ a locally-hacked squash image that has flash-kernel > purged and a PPA kernel that allows grub-efi-arm64 to satisfy the > bootloader Recommends. On an EFI-based system, flash-kernel never gets > installed. On a Moonshot m400, flash-kernel gets installed as expected. > > Once curtin has been updated with a fix like this, and MAAS has been > refreshed to include it, I think we should then be able to remove flash- > kernel from the arm64 cloud images. What's the criteria for determining if a particular arm instance requires flash-kernel or not? Ryan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1918427 Title: curtin: install flash-kernel in arm64 UEFI unexpected To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1918427/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1868177] Re: /snap/subiquity/1561/usr/bin/subiquity:ValueError:main:wrapper:log_time:block_meta:meta_clear:clear_holders:wipe_superblock:_wipe_superblock:wipe_volume:quick_zero:zero_file_at_o
Huh, Never seen partitions smaller than 1MB... Curtin could truncate the requested size to the greater of 1MB or size of partition. On Tue, Feb 2, 2021 at 4:05 PM dann frazier <1868...@bugs.launchpad.net> wrote: > I worked around the MAAS deployment issue by manually removing all > partitions (parted -- mklabel gpt). > > ** Also affects: curtin (Ubuntu) >Importance: Undecided >Status: New > > -- > You received this bug notification because you are subscribed to > subiquity. > Matching subscriptions: curtin-bugs, subiquity-bugs > https://bugs.launchpad.net/bugs/1868177 > > Title: > > /snap/subiquity/1561/usr/bin/subiquity:ValueError:main:wrapper:log_time:block_meta:meta_clear:clear_holders:wipe_superblock:_wipe_superblock:wipe_volume:quick_zero:zero_file_at_offsets > > To manage notifications about this bug go to: > https://bugs.launchpad.net/subiquity/+bug/1868177/+subscriptions > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1868177 Title: /snap/subiquity/1561/usr/bin/subiquity:ValueError:main:wrapper:log_time:block_meta:meta_clear:clear_holders:wipe_superblock:_wipe_superblock:wipe_volume:quick_zero:zero_file_at_offsets To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1868177/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1912844] Re: Bond with OVS bridging RuntimeError: duplicate mac found!
Thanks for doing most of the digging here @Oddbloke; I suspect as with bond and bridges for ovs, we'll need a special case to check if a vlan entry is also OVS, much like we did for bonds/bridges: https://github.com/canonical/cloud-init/pull/608/files So our is_vlan change will need to see if link device is OVS and if so then say it's a vlan as well (since the DEVTYPE doesn't match) or something to that effect. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1912844 Title: Bond with OVS bridging RuntimeError: duplicate mac found! To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1912844/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1908761] Re: Can't deploy using root XFS partition using 18.04
You might just mark this bug duplicate of the other so we can close both if this fixes things; and if so, I can update our vmtest which exercises this path. Note this is broken all the way back through Xenial. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1908761 Title: Can't deploy using root XFS partition using 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1908761/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1908761] Re: Can't deploy using root XFS partition using 18.04
@Eric Yeah, I *think* that will work; I believe at the time I manually performed that step and it worked. I'm not sure w.r.t Secure Boot and shim-signed; Maybe @vorlon can confirm if the xfs grub2 module is signed/included in shim-signed? > As follow: > https://salsa.debian.org/grub-team/grub/-/commit/01f1eee93acd4113475b403f80661cf651fbcc00 Ah, that's adding it to the signed bits... that looks promising. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1908761 Title: Can't deploy using root XFS partition using 18.04 To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1908761/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1907107] Re: cloud-init runs too late at first startup after ubuntu autoinstall
* eoli3n <1907...@bugs.launchpad.net> [2020-12-08 13:30]: > If i run autoinstall with that user-data file, autoinstall doesn't run > and subiquity prompt for language setting. > > #cloud-config > runcmd: > - systemctl daemon-reload > - systemctl restart gdm3 > autoinstall: > version: 1 > packages: > - gdm3 > locale: fr_FR.UTF-8 > user-data: > timezone: Europe/Paris > refresh-installer: > update: yes > identity: > hostname: localhost > username: premier > password: $1$l03JsARH$DCxHdrzUH8SRmeDk3/3rU1 > > If I use that user-data file, runcmd seems not to run at the end of > cloud-config after first reboot. > > #cloud-config > autoinstall: > version: 1 > packages: > - gdm3 > locale: fr_FR.UTF-8 > user-data: > timezone: Europe/Paris > refresh-installer: > update: yes > identity: > hostname: localhost > username: premier > password: $1$l03JsARH$DCxHdrzUH8SRmeDk3/3rU1 > runcmd: > - systemctl daemon-reload > - systemctl restart gdm3 > > How should i use runcmd ? I think part of this confusion is that subiquity has it's autoinstall format[1], which looks rather similar to cloud-config, but it's the same and depending on *where* you need things to run (say in the installer environment, or on firstboot) then you need to adjust the config structure. I believe if you want to pass a user-data section in the autoinstall file[2]. It would look like this: #cloud-config autoinstall: version: 1 locale: fr_FR.UTF-8 refresh-installer: update: yes user-data: locale: fr_FR.UTF-8 hostname: localhost users: - name: premier gecos: premier passwd: "$1$l03JsARH$DCxHdrzUH8SRmeDk3/3rU1" shell: /bin/bash groups: "admin,sudo" lock_passwd: False packages: - gdm3 timezone: Europe/Paris runcmd: - systemctl daemon-reload - systemctl restart gdm3 The autoinstaller will merge the 'user-data' section of the autoinstall cloud-config it normally generates and then place this config in the target OS so that in first boot cloud init will do all of the things you need. > > FYI, even if i restart gmd3 manually with Fr locale set and generated, > it is still in english. Then only timezone test is ok, if i > restart manually after cloud-config, clock is well configured. Looking at the config I think if you use the user-data key in autoinstall conf you can be sure that all of the required bits (locale, timezone, packages, etc) do what's needed. 1. https://ubuntu.com/server/docs/install/autoinstall-reference 2. https://github.com/CanonicalLtd/subiquity/blob/main/examples/autoinstall-user-data.yaml Ryan -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1907107 Title: cloud-init runs too late at first startup after ubuntu autoinstall To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1907107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1907107] Re: cloud-init runs too late at first startup after ubuntu autoinstall
* eoli3n <1907...@bugs.launchpad.net> [2020-12-08 11:50]: > > I can say that the Ubuntu Server installer isn't typically used to install > > Ubuntu Desktop. > I use it because i can't find any netboot installer which supports > autoinstall and EFI installation. I used legacy netboot (60MB) for my > kickstart install but now i need to autoinstall an EFI system. OK > > > have you looked at just using the Ubuntu Desktop installer with OEM > > mode? > > https://help.ubuntu.com/community/Ubuntu_OEM_Installer_Overview > I just want a way to automate installation from a pxe boot, with an > "answer file", i did with kickstart in a "script" model, i want to do > the same with autoinstall. Sure > > cloud-init is configured to write to the console by default; that's > > why you see output from cloud-init on your tty. > It seems to me a little bit buggy, don't you think ? Well, that's not a bug. It's the default configuration for cloud-init as it runs in places where there's no access to tty or console; just the output that a cloud platform may show you. You may want to remove/override the StandardOutput setting in the cloud-init service units (cloud-init-local, cloud-init, cloud-config, cloud-final). The default is to log to both journal and console. # Output needs to appear in instance console output StandardOutput=journal+console It's something that subiquity could configure differently since in this scenario, the target system would have user-facing console. > I think cloud-config.service should defaulty have a stronger > dependency as Before=getty.target for exemple. Possibly. If you want to test that out, you can add this as a drop-in conf, before for boot the target OS you can: mkdir -p /target/lib/systemd/system/cloud-config.service.d echo -e "[Unit]\nBefore=getty.target > /target/lib/systemd/cloud-config.service.d/10-before-getty.conf > > Why don't the installer configure everything during the install process > ? Generating locales, write /etc/default/locale and set /etc/timezone > are not hard tasks. Even if cloud-init embeed those configuration, you > could tell the installer to setup what in cloud-config as default > config. The design is based on the idea of a template image. The Ubuntu server image is ready to boot and on first boot cloud-init handles the tasks to "instantiate the image". Doesn't the autoinstall input generatea cloud-init config that configures these things? > > If you can't avoid that first startup scripts, you should start as early > as possible in the init process, before any process that could use what > it setup/modify. Does that seem illogical ? It's not. Cloud-init does start up as early as possible (after local fs is mounted). However some tasks require to be run later; for example package install cannot run without networking being configured. At this point, I don't think there is a bug in cloud-init. For your use-case, you may need change some of the defaults that are already cofigured in the image. It's probably worth looking at your working solution to see what, if anything, could be configured by subiquity; or if it makes sense to tweak a default. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1907107 Title: cloud-init runs too late at first startup after ubuntu autoinstall To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1907107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1907107] Re: cloud-init runs too late at first startup after ubuntu autoinstall
Can you run and attach the output tarball on the target system where you see this fail: cloud-init collect-logs With respect to your use-case; cloud-init does not directly control what packages do at install time. Some package may require post-install configuration, or are not typically installed during the boot process. Just testing your config, installing lightdm brings in over 500 packages and took about 12 minutes to install; This is a bit of an uphill struggle; I can say that the Ubuntu Server installer isn't typically used to install Ubuntu Desktop; have you looked at just using the Ubuntu Desktop installer with OEM mode? https://help.ubuntu.com/community/Ubuntu_OEM_Installer_Overview I suspect you'll need to employ some additional changes to the cloud- config to handle these edge cases; I don't think the respective packages, like gdm, would change. I would suggest that you add a runcmd to restart gdm or other packages which might need to restart after the initial install; reload any installed systemd units with daemon-reload and then restart gdm. runcmds are executed after package install. runcmd: - systemctl daemon-reload - systemctl gdm restart > If i switch to another tty, then i can see cloud-config running, messing up tty login prompt cloud-init is configured to write to the console by default; that's why you see output from cloud-init on your tty. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1907107 Title: cloud-init runs too late at first startup after ubuntu autoinstall To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1907107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1907107] Re: cloud-init runs too late at first startup after ubuntu autoinstall
Please move this back to New after you've attached the cloud-init collect-logs output tarball. ** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1907107 Title: cloud-init runs too late at first startup after ubuntu autoinstall To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1907107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1907107] Re: cloud-init runs too late at first startup after ubuntu autoinstall
I'm marking the cloud-init task invalid for now. As mentioned in the other bug[1]; cloud-init is not running too late; rather scripts not provided to cloud-init are not waiting for cloud-init to complete its execution. If your scripts are using cloud-init status --wait before executing or systemd units are gated with After=cloud- init.target, then please re-open this task by marking it New again. When doing so please attach ubuntu-bug cloud-init output and a sample of your scripts and how they're added to the image so we can see when they are running. I do believe that if you gate your scripts to run after cloud-init has completed there shouldn't be any issues. 1. https://bugs.launchpad.net/subiquity/+bug/1906829/comments/11 ** Changed in: cloud-init (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1907107 Title: cloud-init runs too late at first startup after ubuntu autoinstall To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1907107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1906187] [NEW] Version tag is not respected when put last
* Christian Dohm <1906...@bugs.launchpad.net> [2020-11-29 15:50]:
> Public bug reported:
>
Thank you for filing a bug.
> I've created a 'network-config' file with Terraform's yamldecode() function
> that contains (btw. I've tried with the version being a Number w/o quotes and
> as well as a String as shown here):
> ---
> "network":
> "ethernets":
> "eth0":
> "gateway4": "192.168.1.1"
> "nameservers":
> "addresses":
> - "192.168.1.74"
> - "192.168.1.104"
> "search":
> - "fritz.box"
> "set-name": "eth0"
> "version": "2"
> ---
> After running on Raspberry Pi 4B with 4 GB, created with
> ubuntu-20.04.1-preinstalled-server-arm64+raspi.img.xz, the machine's setup
> fails and /var/log/cloud-init.log reveals that:
> ---
> 2020-04-01 17:23:48,649 - util.py[DEBUG]: Reading from
> /boot/firmware//network-config (quiet=False)
> 2020-04-01 17:23:48,649 - util.py[DEBUG]: Read 245 bytes from
> /boot/firmware//network-config
> 2020-04-01 17:23:48,650 - util.py[DEBUG]: Attempting to load yaml from string
> of length 240 with allowed root types (,)
> 2020-04-01 17:23:48,652 - util.py[DEBUG]: Attempting to load yaml from string
> of length 245 with allowed root types (,)
> 2020-04-01 17:23:48,656 - DataSourceNoCloud.py[DEBUG]: Top level network key
> in network-config but missing 'config' or 'version': {'network':
> {'ethernets': {'eth0': {'gateway4': '192.168.1.1', 'nameservers':
> {'addresses': ['192.168.1.74', '192.168.1.104'], 'search': ['fritz.box']},
> 'set-name': 'eth0'}}, 'version': '2'}}
This happens due to the _maybe_remove_top_network method which only
understood network-config v1 format. If the config provided included
the 'network' key, it would remove it. It does some additional
checks (for v1 only) and if they don't match it returns the config
as it was loaded; in this case, with the top-level 'network' key.
Then this network config is passed to cloudinit.net which expects
the DataSource's .network_config property to include a config without a
'network' key; however, in this case, it still has the top-level key
and this confuses the .net's extract_physdevs function.
We should fix _maybe_remove_top_network to handle either v1 or v2;
We may want to accept version field as string and attempt to int() the
value.
As a workaround, if you specify your network config version 2 with the
following changes:
1) remove 'network' key
2) use integer 2 in the 'version' field
3) optionally, your 'set-name' requires use of a 'match' attribute,
otherwise it's ignored.
set-name: eth0
match:
macaddress: 'AA:BB:CC:DD:EE:FF'
This should let you progress.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1906187
Title:
Version tag is not respected when put last
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1906187/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899993] Re: EFI: Fails when BootCurrent entry does not exist
@nb This issue is *slightly* different than the other bug related to
missing BootCurrent value.
The previous bug which is fixed in curtin happens when the BootCurrent
value is not present in the efibootmgr output.
This bug has a BootCurrent value, like 0003, however, the Entries
provided by efibootmgr do not have one for 0003.
Here's the raw output from efibootmgr -v:
BootCurrent: 0003
Timeout: 10 seconds
BootOrder: 0003,0004,0005,0006,0001
This system has no boot entries; this is a firmware bug.
>From the log file, after curtin adds an entry for Ubuntu we can see the
boot entries look like this:
{
"current": "0003",
"timeout": "10 seconds",
"order": [
""
],
"entries": {
"": {
"name": "ubuntu",
"path":
"HD(1,GPT,0937ffdf-628c-4161-8b2f-5920235669c6,0x800,0x10)/File(\\EFI\\ubuntu\\shimx64.efi)"
}
}
}
Current pointed entry '0003', however, the list of entries only contains
'' (what curtin just installed). Then when curtin attempts to place
the newly installed entry at the beginning of the BootOrder list, it
fails when we run this command:
efibootmgr -o 0003,
Invalid BootOrder order entry value0003
^
efibootmgr: entry 0003 does not exist
The fix for this in curtin is to verify that BootCurrent points to an existing
entry; if not, then we ignore the BootCurrent value.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/183
Title:
EFI: Fails when BootCurrent entry does not exist
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/183/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899993] Re: EFI: Fails when BootCurrent entry does not exist
@Dann For this firmware bug where we have missing entries; Have you ever booted this up and had efibootmgr show all of the entries? Looking at this output here: BootCurrent: 0003 Timeout: 10 seconds BootOrder: ,0003,0004,0005,0006,0001 Boot* ubuntu HD(1,GPT,0937ffdf-628c-4161-8b2f-5920235669c6,0x800,0x10)/File(\EFI\ubuntu\shimx64.efi) With my suggested change to ignore BootCurrent since it does not have a matching entry; curtin can at best put the new entry first in order (note this is already done by grub-install). This does mean that the node will be lost from maas control since it is no longer PXE booting... Is that acceptable? ** Changed in: curtin (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/183 Title: EFI: Fails when BootCurrent entry does not exist To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/183/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899993] Re: EFI: Fails when BootCurrent entry does not exist
@Dann OK. That sounds reasonable. For this issue, curtin can (and I think it makes sense) ignore a BootCurrent value if there is no corresponding entry; warn in the logs that it's missing, check your firmware, etc... and proceed if it can. ** Changed in: curtin (Ubuntu) Importance: Undecided => Medium ** Changed in: curtin (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/183 Title: EFI: Fails when BootCurrent entry does not exist To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/183/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time
On Thu, Oct 15, 2020 at 10:45 AM Frode Nordahl <1899...@bugs.launchpad.net> wrote: > Ryan, thanks for those pointers, will check. I also see in #15 that > Bionic uses Fallback while Focal uses an actual ds, don't know why > though. > Bah, I *keep* forgetting, that Bionic does *NOT* read OpenStack metadata service by default https://github.com/canonical/cloud- init/commit/cd1de5f47ab6b82f2c6fd61a5f6681f33b3e5705 That landed right *after* 18.04 was released. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899487 Title: cloud-init hard codes MTU configuration at initial deploy time To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899993] Re: EFI: Fails when BootCurrent entry does not exist
Looking back at your patch:
Index: curtin-20.1-29-g81144052/curtin/util.py
===
--- curtin-20.1-29-g81144052.orig/curtin/util.py
+++ curtin-20.1-29-g81144052/curtin/util.py
@@ -886,6 +886,7 @@ def get_efibootmgr(target=None):
"""
with ChrootableTarget(target=target) as in_chroot:
stdout, _ = in_chroot.subp(['efibootmgr', '-v'], capture=True)
+LOG.info("DANNF:\n%s", stdout)
output = parse_efibootmgr(stdout)
return output
And the logs:
Running command ['unshare', '--fork', '--pid', '--', 'chroot',
'/tmp/tmp6ha4_iz2/target', 'efibootmgr', '-v'] with allowed return codes [0]
(capture=True)
DANNF:
BootCurrent: 0003
Timeout: 10 seconds
BootOrder: 0003,0004,0005,0006,0001
The firmware appears to have no entries at all; but somehow has a BootCurrent
and a BootOrder pointing to entries that don't exist. This certainly has to be
a firmware bug no?
** Changed in: curtin (Ubuntu)
Status: New => Incomplete
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/183
Title:
EFI: Fails when BootCurrent entry does not exist
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/183/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time
> I guess it's time for me to ask a question: is it cloud-init that
> renders /etc/netplan/50-cloud-init.yaml? If so where does netplan
> fit in when the difference is how that file is rendered and not
> how it is interpreted. As you can see in #10 the mtu statement is
> not in the file on bionic, while it is on focal.
>
> Since versions appear to be the same my guess would be that there
> is some internal modelling of how bionic vs. focal should be
> configured?
There isn't an internal model; cloud-init SRU's master back to
previous releases. For OpenStack, cloud-init on Xenial does not
render network-data.json by default as that's a behavior change
added in Bionic and newer.
The pipeline looks like:
cloudinit (fetch network-data.json from OpenStack)
`-> cloudinit (convert network-data.json to network-config-v1)
`-> cloudinit (converts network-config-v1 -> netplan on Ubuntu)
`-> cloudinit (calls netplan generate -> systemd-networkd files)
> I can't let go of why the same version of cloud-init renders
> different config with the same data source.
Are you sure the 'mtu' value it was present in the network-data.json
at the time cloud-init fetched it vs. when you curl later?
Ceck the cloud-init.log file; you should see the network
config after it was converted from network-data.json somewhere
in the log.
Give your JSON from [10], bionic and focal render this the same
## BIONIC ##
% lxc launch ubuntu-daily:bionic b1
% lxc exec b1 bash
root@b1:~# lsb_release -rd
Description:Ubuntu 18.04.5 LTS
Release:18.04
root@b1:~# dpkg --list | egrep "(cloud-init|netplan)"
ii cloud-init 20.3-2-g371b392c-0ubuntu1~18.04.1 all
Init scripts for cloud instances
ii cloud-initramfs-copymods 0.40ubuntu1.1 all
copy initramfs modules into root filesystem for later use
ii cloud-initramfs-dyn-netconf0.40ubuntu1.1 all
write a network interface file in /run for BOOTIF
ii libnetplan0:amd64 0.99-0ubuntu3~18.04.3 amd64
YAML network configuration abstraction runtime library
ii netplan.io 0.99-0ubuntu3~18.04.3 amd64
YAML network configuration abstraction for various backends
root@b1:~# cat /etc/cloud/build.info
build_name: server
serial: 20201014
root@b1:~# cat network-data.json
{"links": [{"id": "tapc352887e-0f", "vif_id":
"c352887e-0fff-481b-af47-7df9f7c2ff05", "type": "ovs", "mtu": 8950,
"ethernet_mac_address": "fa:16:3e:a3:34:78"}], "networks": [{"id": "network0",
"type": "ipv4_dhcp", "link": "tapc352887e-0f", "network_id":
"f8123ceb-e29d-4f4a-b200-6fb3bf3984ba"}], "services": []}
root@b1:~# cloud-init devel net-convert --network-data network-data.json -k
network_data.json -m "ens4,fa:16:3e:a3:34:78" -d test -D ubuntu --debug -O
netplan
2020-10-15 15:09:17,796 - util.py[DEBUG]: Reading from /proc/uptime
(quiet=False)
2020-10-15 15:09:17,797 - util.py[DEBUG]: Read 14 bytes from /proc/uptime
2020-10-15 15:09:17,797 - util.py[DEBUG]: Reading from
/sys/class/net/eth0/addr_assign_type (quiet=False)
2020-10-15 15:09:17,797 - util.py[DEBUG]: Read 2 bytes from
/sys/class/net/eth0/addr_assign_type
2020-10-15 15:09:17,797 - util.py[DEBUG]: Reading from
/sys/class/net/eth0/uevent (quiet=False)
2020-10-15 15:09:17,797 - util.py[DEBUG]: Read 26 bytes from
/sys/class/net/eth0/uevent
2020-10-15 15:09:17,797 - util.py[DEBUG]: Reading from
/sys/class/net/eth0/address (quiet=False)
2020-10-15 15:09:17,797 - util.py[DEBUG]: Read 18 bytes from
/sys/class/net/eth0/address
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from
/sys/class/net/eth0/device/device (quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from
/sys/class/net/lo/addr_assign_type (quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Read 2 bytes from
/sys/class/net/lo/addr_assign_type
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from /sys/class/net/lo/uevent
(quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Read 23 bytes from
/sys/class/net/lo/uevent
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from
/sys/class/net/lo/address (quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Read 18 bytes from
/sys/class/net/lo/address
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from
/sys/class/net/lo/device/device (quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from /sys/class/net/eth0/type
(quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Read 2 bytes from
/sys/class/net/eth0/type
2020-10-15 15:09:17,798 - util.py[DEBUG]: Reading from /sys/class/net/lo/type
(quiet=False)
2020-10-15 15:09:17,798 - util.py[DEBUG]: Read 4 bytes from
/sys/class/net/lo/type
Internal State
--- !!python/object:cloudinit.net.network_state.NetworkState
_has_default_route: null
_network_state:
config:
- mac_address: fa:16:3e:a3:34:78
mtu: 8950
name: ens4
subnets:
- type: dhcp4
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Thanks for the logs Dann, The error comes from efibootmgr itself; so I don't think this is the same issue. You can open a new bug; but it still looks like a firmware/platform issue. efibootmgr claims there are 5 entries; and at least after a grub install, there is only one. What happened to all of those entries? BootCurrent: 0003 Timeout: 10 seconds BootOrder: ,0003,0004,0005,0006,0001 Boot* ubuntu HD(1,GPT,0937ffdf-628c-4161-8b2f-5920235669c6,0x800,0x10)/File(\EFI\ubuntu\shimx64.efi) This bug and curtin and the change that's merged prevents curtin from removing the BootCurrent entry, 0003; which it did not; otherwise we'd see: Removing duplicate EFI entry -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time
> On the flip side the presence of the MTU key in the OpenStack > metadata cannot be used as an indicator for intent from either the > system or the user that the DHCP server should not be providing the > MTU either. > > Looking at the commit that changed the behaviour in OpenStack the > intent of the original code was to always provide the MTU value in > the metadata regardless of network type, the fact that it showed as > `null` was a bug. > > Up until 2017 the default for the OpenStack controlled DHCP server > was to always provide an MTU. In 2017 the ability to control this > behaviour was removed and from that point onward it always provides > an MTU. > > The user has no way of influencing the contents of the OpenStack > network metadata, apart from downgrading to a 5 year old version. > Before we continue suggesting that cloud-init should somehow guess what OpenStack meant to do with network configuration it provided to cloud-init I'd like to make sure we discuss what we're suggesting. If we can guess that an OpenStack which sent an MTU really didn't mean for cloud-init to use this MTU because the DHCP server might also send an MTU, can we not guess that the IP address it sent us wasn't the correct one and instead we should add .1 to the lowest octet? I'm being pendantic here to make a point. OpenStack is the "Oracle" here; just like MAAS, or Ec2 or Azure. The network configuration it provides to the guest is meant to be taken as it is provided. If the configuration is sub-optimal should not the cloud itself resolve this? Has there been any attempt to ask in OpenStack upstream why the MTU plumbing/control was removed? If a specific MTU is needed for a network in OpenStack, should that not be configured in OpenStack such that the MTU value is either statically provided in the network-data.json sent *or* update the DHCP server running on that network to provide the correct value? > I don't see an easy way of overriding cloud-inits default behaviour > by adding additional configuration through vendor data either. This is correct. Network-config cannot be part of user-data or vendor-data; the network needs to be configured prior to cloud-init fetching/reading this data (which may be URLs to remote cloud-config). > > Perhaps adding a cloud-init config stanza for how the OpenStack > source driver should interpret the presence of MTU in the network > metadata could be a path to retain compability with anyone relying > on the current behaviour and at the same time providing a way > forward for everyone else? > > Meanwhile instances are configured to obtain an address dynamically > but stuck with a static value for MTU forever, and not being able to > adjust to changes being made to the environment without manual > intervention to individual instances. If the network-data.json provided by the metadata service is not sufficient and you cannot change this service one can provide a network configuration in a ConfigDrive. ** Changed in: cloud-init Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899487 Title: cloud-init hard codes MTU configuration at initial deploy time To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time
> So I would suggest that whenever OpenStack eludes to dynamic configuration > being in play cloud-init should not write the MTU value into the on-disk > configuration but let it be configured by dynamic network configuration > protocol. > > What do you think? I would argue the opposite. The existing behavior is that the MTU provided by the network-data.json is the source of truth. Cloud-init itself cannot determine whether the DHCP service provides MTU or not, nor whether the MTU provided in the DHCP response is infact the desired MTU. Further, for cloud-init now to start ignoring MTU if DHCP is present would break existing users who's DHCP server either does not provide MTU or provides an incorrect value. If network-data.json MTU value is null, then I think it all of this works the way you want, correct? Looking at the bug which fixed this always null value should it not be enhanced to only set this MTU value if the DHCP service on the network is not providing an MTU? ** Changed in: cloud-init Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1899487 Title: cloud-init hard codes MTU configuration at initial deploy time To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1899487] Re: cloud-init hard codes MTU configuration at initial deploy time
" # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
{"links": [{"id": "tapa035fb68-01", "vif_id":
"a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942,
"ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0",
"type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id":
"b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}"
How is cloud-init to know from this network-config.json that DHCP will
provide an MTU value? How does it know that it should ignore the
provided MTU? If DHCP is providing MTU, should network-config.json then
not provide the MTU value?
** Also affects: netplan.io (Ubuntu)
Importance: Undecided
Status: New
** Changed in: cloud-init
Status: New => Incomplete
** Changed in: netplan.io (Ubuntu)
Status: New => Incomplete
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1899487
Title:
cloud-init hard codes MTU configuration at initial deploy time
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1899487/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
@Dann Thanks. Can you provide the log without your changes? Or at least the efibootmgr dump before curtin starts processing things? > Notice we have no entry for current/0003. I have not inspected the code to > determine whether or not this is an issue parsing efibootmgr output or the > actual variable state. Is this not the bug? The full logs will show what efibootmgr -v shows. I that state is not good (BootCurrent is set, but no entry to match) then that's firmware bug (IMHO). If current entry is present then we likely do have a parsing bug. Either way the full log with efibootmgr -v dump in the logs will shed light on what to do next. ** Changed in: curtin (Ubuntu) Status: Confirmed => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
@Jeff I cannot read the canonical pastebin output, can you use paste.ubuntu ? re: -updates; MAAS needs to initiate an SRU for archive users of MAAS. @nb I don't know for sure. I was told that MAAS snaps would pull curtin from git master (or the respective release branches). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1895922] Re: maas could not deploy a kvm-based vm via virsh
Hi Patricia, You'll need to upgrade MAAS itself. The curtin_userdata is what's sent to the nodes deploying. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1895922 Title: maas could not deploy a kvm-based vm via virsh To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1895922/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1895922] Re: maas could not deploy a kvm-based vm via virsh
I believe this is a duplicate of: https://bugs.launchpad.net/curtin/+bug/1876258 This has already been fixed in curtin, you can test with curtin's daily PPA, or curtin in groovy. W.r.t uvtool; like MAAS, uvtool should generate serial disk attributes for all virtio disks by default. ** Changed in: curtin Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1895922 Title: maas could not deploy a kvm-based vm via virsh To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1895922/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
@Seth Thanks for verifying! -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1895044] Re: update_nvram defaults to false, not true as per docs
@ltrager Any idea why the maas verification for the 20.1 SRU didn't catch this? curtin's vmtest runs with pristine NVRAM, so we always run with update_nvram=true otherwise we can't boot into whatever is installed. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1895044 Title: update_nvram defaults to false, not true as per docs To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1895044/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1895044] Re: update_nvram defaults to false, not true as per docs
mwhudson already saw this and fixed in master, though without filing a bug and no review AFAICT. =( https://git.launchpad.net/curtin/commit/?id=83944d61e95ea7eb17289625330f0863bec39488 So daily curtin groovy is fixed. This is released in 20.1 so we'll need to SRU to Xenial, Bionic, Focal. ** Changed in: curtin Status: New => Fix Committed ** Also affects: curtin (Ubuntu) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Focal) Importance: Undecided Status: New ** Changed in: curtin (Ubuntu Groovy) Status: New => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1895044 Title: update_nvram defaults to false, not true as per docs To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1895044/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Hi Jeff, That's great. I'm just pushing this into a ppa for testing: https://launchpad.net/~raharper/+archive/ubuntu/lp1894217 Groovy should publish any minute now, and I'll copy into focal/bionic/xenial pockets shortly after. Please let me know if this fixes things. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1888726] Re: systemd-udevd regression: some renamed network interfaces stuck in "pending" state
@Dan Just ran into this issue in curtin's vmtest for network_vlan on groovy. I can confirm that if the netplan config includes the driver match on the physical interfaces, then the vlans come up just fine. I was thinking that netplan could inject the driver of the underlying device into the match section. That would help in these situations, however, I know of a few scenarios where this would need to be disabled (On Azure, for example, their Advanced Networking which auto-bonds an SRIOV interface and a HyperV nic together as eth0). Also, this scenario does *NOT* fail for me on Focal, so I'm wondering what changed between either netplan or systemd? Focal: netplan.io 0.99-0ubuntu3~20.04.2 systemd 245.4-4ubuntu3.2 Groovy netplan.io 0.99-0ubuntu6 systemd 246-2ubuntu1 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1888726 Title: systemd-udevd regression: some renamed network interfaces stuck in "pending" state To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/netplan.io/+bug/1888726/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894910] Re: fallocate swapfile has holes on 5.8 ext4
Turns out it's unrelated to bcache; it is trivially reproducible: lxc launch ubuntu-daily:groovy g1 --vm lxc exec g1 bash fallocate -l 1024M /swap.img mkswap /swap.img swapon --verbose /swap.img cat /proc/swaps On the 5.4 kernel that groovy had a few weeks back this works, on daily (5.8) this fails: root@g2vm:/home/ubuntu# uname -r 5.8.0-18-generic root@g2vm:/home/ubuntu# fallocate -l 1024M /swap.img root@g2vm:/home/ubuntu# mkswap /swap.img mkswap: /swap.img: insecure permissions 0644, 0600 suggested. Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes) no label, UUID=dafabd56-1137-459c-8e61-b6335eac1d1a root@g2vm:/home/ubuntu# swapon --verbose /swap.img swapon: /swap.img: insecure permissions 0644, 0600 suggested. swapon: /swap.img: found signature [pagesize=4096, signature=swap] swapon: /swap.img: pagesize=4096, swapsize=1073741824, devsize=1073741824 swapon /swap.img swapon: /swap.img: swapon failed: Invalid argument root@g2vm:/home/ubuntu# cat /proc/swaps FilenameTypeSizeUsed Priority root@g2vm:/home/ubuntu# dmesg | tail -n 2 [7.778887] EXT4-fs (sda1): resized filesystem to 2412985 [ 32.766796] swapon: swapfile has holes Here's groovy from a few weeks back root@g1fat:/home/ubuntu# cat /etc/cloud/build.info build_name: server serial: 20200819.1 root@g1fat:/home/ubuntu# uname -r 5.4.0-42-generic root@g1fat:/home/ubuntu# cat /proc/swaps FilenameTypeSizeUsedPriority root@g1fat:/home/ubuntu# fallocate -l 1024M /swap.img root@g1fat:/home/ubuntu# mkswap /swap.img mkswap: /swap.img: insecure permissions 0644, 0600 suggested. Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes) no label, UUID=aef79113-dd6b-4029-b558-f5269150d853 root@g1fat:/home/ubuntu# swapon --verbose /swap.img swapon: /swap.img: insecure permissions 0644, 0600 suggested. swapon: /swap.img: found signature [pagesize=4096, signature=swap] swapon: /swap.img: pagesize=4096, swapsize=1073741824, devsize=1073741824 swapon /swap.img root@g1fat:/home/ubuntu# dmesg | tail -n 2 [431543.292314] Adding 1048572k swap on /swap.img. Priority:-2 extents:5 across:1187840k FS root@g1fat:/home/ubuntu# cat /proc/swaps FilenameTypeSizeUsedPriority /swap.img file1048572 0 -2 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894910 Title: fallocate swapfile has holes on 5.8 ext4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1894910/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894910] Re: fallocate swapfile has holes on 5.8 ext4
** Summary changed: - fallocate swapfile has holes on 5.8 ext4 over bcache + fallocate swapfile has holes on 5.8 ext4 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894910 Title: fallocate swapfile has holes on 5.8 ext4 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1894910/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894910] [NEW] fallocate swapfile has holes on 5.8 ext4 over bcache
Public bug reported: 1) Groovy 2) root@ubuntu:/home/ubuntu# apt-cache policy linux-image-generic linux-image-generic: Installed: 5.8.0.18.22 Candidate: 5.8.0.18.22 Version table: *** 5.8.0.18.22 500 500 http://archive.ubuntu.com/ubuntu groovy/main amd64 Packages 100 /var/lib/dpkg/status 3) swapon --verbose /swap.img enables swap file successfully 4) swapon returns error, says swapfile has holes. Curtin creates file-based swap files with fallocate: fallocate -l XXXM /swap.img When booting Groovy (5.8 kernel), the swapfile fails to load, complains it has holes root@ubuntu:/home/ubuntu# swapon --verbose /swap.img swapon: /swap.img: found signature [pagesize=4096, signature=swap] swapon: /swap.img: pagesize=4096, swapsize=1251999744, devsize=1251999744 swapon /swap.img [ 178.283022] swapon: swapfile has holes swapon: /swap.img: swapon failed: Invalid argument root@ubuntu:/home/ubuntu# uname -r 5.8.0-18-generic This configuration works fine on Focal (5.4) and older. Previous Groovy kernels, (5.7) also worked. It may be related to: https://lkml.org/lkml/2020/9/4/259 ProblemType: Bug DistroRelease: Ubuntu 20.10 Package: linux-image-generic 5.8.0.18.22 ProcVersionSignature: Ubuntu 5.8.0-18.19-generic 5.8.4 Uname: Linux 5.8.0-18-generic x86_64 AlsaDevices: total 0 crw-rw 1 root audio 116, 1 Sep 8 21:38 seq crw-rw 1 root audio 116, 33 Sep 8 21:38 timer AplayDevices: Error: [Errno 2] No such file or directory: 'aplay' ApportVersion: 2.20.11-0ubuntu44 Architecture: amd64 ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord' AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1: CasperMD5CheckResult: skip Date: Tue Sep 8 21:44:13 2020 IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig' Lsusb: Error: command ['lsusb'] failed with exit code 1: Lsusb-t: Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1: MachineType: QEMU Standard PC (i440FX + PIIX, 1996) PciMultimedia: ProcEnviron: TERM=vt220 PATH=(custom, no user) LANG=C.UTF-8 SHELL=/bin/bash ProcFB: 0 bochs-drmdrmfb ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-5.8.0-18-generic root=UUID=ce45bbaf-5a44-4487-b89e-035c2dd40657 ro console=ttyS0 systemd.mask=snapd.seeded.service systemd.mask=snapd.service panic=-1 softlockup_panic=1 hung_task_panic=1 nmi_watchdog=panic,1 ip=:BOOTIF:dhcp RelatedPackageVersions: linux-restricted-modules-5.8.0-18-generic N/A linux-backports-modules-5.8.0-18-generic N/A linux-firmware1.190 RfKill: Error: [Errno 2] No such file or directory: 'rfkill' SourcePackage: linux UpgradeStatus: No upgrade log present (probably fresh install) dmi.bios.date: 04/01/2014 dmi.bios.release: 0.0 dmi.bios.vendor: SeaBIOS dmi.bios.version: 1.13.0-1ubuntu1 dmi.chassis.type: 1 dmi.chassis.vendor: QEMU dmi.chassis.version: pc-i440fx-focal dmi.modalias: dmi:bvnSeaBIOS:bvr1.13.0-1ubuntu1:bd04/01/2014:br0.0:svnQEMU:pnStandardPC(i440FX+PIIX,1996):pvrpc-i440fx-focal:cvnQEMU:ct1:cvrpc-i440fx-focal: dmi.product.name: Standard PC (i440FX + PIIX, 1996) dmi.product.version: pc-i440fx-focal dmi.sys.vendor: QEMU ** Affects: linux (Ubuntu) Importance: Undecided Status: Confirmed ** Tags: amd64 apport-bug curtin-vmtest groovy uec-images -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894910 Title: fallocate swapfile has holes on 5.8 ext4 over bcache To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1894910/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Thanks for the logs. They certainly seem to be duplicate entries, so AFAICT, the removal seems reasonable. However, removing BootCurrent shouldn't happen as we plan to boot back into that entry anyhow. I've got a branch that I'll build and put into a PPA so you can test here shortly. It does two things: 1) does not remove BootCurrent if it is a duplicate 2) adds a new curtin grub config to enable users to override the duplicate removal behavior -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Thanks for the logs. ** Changed in: curtin (Ubuntu) Status: Incomplete => In Progress ** Changed in: curtin (Ubuntu) Assignee: (unassigned) => Ryan Harper (raharper) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Hi David, would you be able to attach: 1) efibootmgr -v output (with all of the network entries present? it appears that there may be some duplicate entries in the menu; I'd like to confirm that by examining verbose output. Specifically Boot -> Boot0003. Boot* EFI Network 1 Boot0001* EFI Network 2 Boot0002* EFI Network 3 Boot0003* EFI Network 4 2) could you attach the deploy log from your successful deployment? ** Changed in: curtin (Ubuntu) Status: Triaged => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1894217 Title: 2.8.2 deploy and commission fails corrupted bootorder variable detected To manage notifications about this bug go to: https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1894217] Re: 2.8.2 deploy and commission fails corrupted bootorder variable detected
Thanks for filing this bug and linking to the other EFI related one. They
aren't the same.
This looks like a bug in curtin's remove duplicate code:
Removing duplicate EFI entry (0006, {'name': 'EFI Network 1', 'path':
'PcieRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x0)/MAC(ecf4bbcdee54,1)/IPv4(0.0.0.00.0.0.0,0,0)'})
Removing duplicate EFI entry (0007, {'name': 'EFI Network 2', 'path':
'PcieRoot(0x0)/Pci(0x1,0x0)/Pci(0x0,0x1)/MAC(ecf4bbcdee55,1)/IPv4(0.0.0.00.0.0.0,0,0)'})
Those entries aren't dupicates, the macaddr in the 'path' element
differ. And, I don't think we should ever attempt to remove the the
BootCurrent value. And, there should be a curtin.grub config option to
enable/disable remove duplicates.
** Also affects: curtin (Ubuntu)
Importance: Undecided
Status: New
** Changed in: curtin (Ubuntu)
Importance: Undecided => Medium
** Changed in: curtin (Ubuntu)
Status: New => Confirmed
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1894217
Title:
2.8.2 deploy and commission fails corrupted bootorder variable
detected
To manage notifications about this bug go to:
https://bugs.launchpad.net/maas/+bug/1894217/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
> This commit adds VROC container and array creation (on clean disks) > https://github.com/gyurco/curtin/commit/fd72c17665c071cde3eb5e047662b04ab993a0dd Nice! Now here's the not so fun part. We've not yet moved curtin to github, so code submissions are done here: https://curtin.readthedocs.io/en/latest/topics/hacking.html -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
> I think your suggestion is a good YAML scheme. I think size_kb: > should be optional to fill the whole array with one volume if > it's omitted. Well, it's not that simple. What do we do if it's omitted and the config includes multiple volumes from the same container? Curtin config is typically constructed from an "Oracle"; either MAAS or Subiquity probe storage and then provide complete configuration from user-input. So if either added support for VROC, they would know in advance how many volumes per container and could specify the exact size. I believe we want to require size_kb if metadata == 'imsm' > The number of devices can be looked up by pairing > 'container' with 'id'. Yes, I put the id anchor there so that the volumes created with in a container will be able to lookup the correct value rather than having to repeat. > Maybe it would be possible to abstract out the container, like: > - type: raid Everything needs and id. id: raid_container_0 > metadata: imsm > container: /dev/md/imsm0 This isn't needed, IIUC, metadata=imsm implies a container raid, no? > name: imsm0 name: /dev/md/imsm0 > devices: > - /dev/nvme0n1p1 > - /dev/nvme1n1p1 > > - type: raid > devices: > - /dev/md/imsm0 # but need to get the number of real devices for -n Yes, this is nice, and what we will do instead is: devices: - raid_container_0 > name: mirror0 > level: 1 > I suggest to add a level: container to the top-level, as it would > imply to use the -e switch to mdadm, and also would be consistent > to the query output. > > - type: raid > id: disk_raid_container0 > level: container > metadata: imsm I think we can skip that if it's true that imsm is always a container. > name: /dev/md/imsm0 > devices: > - /dev/nvme0n1p1 > - /dev/nvme1n1p1 > Another buglet: metadata is not passed to mdadm_create() in > raid_handler() Good catch! And in that case we can always pass --metadata= to mdadm, if metadata is not provided, we use the default. Containers, of course, require metadata=imsm -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
> 1-2) No, only the md-device, the container, raid level and number of devices (it's possible to create different RAID volumes with different levels in one container, however the number of devices must be the same in all). OK. Looking at the https://www.intel.com/content/dam/support/us/en/documents/memory-and-storage/ssd-software/Linux_VROC_6-0_User_Guide.pdf without specifying a size, (raid never does, it uses whole devices) I can't quite wrap my head around what creating to volumes in a container means. If I have 4 disks, and put them into a container and then create a raid0 and a raid5 from the container ... what devices do I have? Ah, the manual suggests size is important for the multi-container support -l Specifies the RAID level. The options supported are 0, 1, 5, 10. -z Specifies the size (in Kibibyte) of space dedicated on each disk to the RAID volume. This must be a multiple of the chunk size. For example > 3) AFAIK after stopping and removing all arrays from the container, mdamd --zero-superblock on the member devices will remove the container itself. Need to check again. OK. We currently do: a) enumerate devices and spares b) set_sync_action=idle c) set_sync_action=frozen d) wipe the superblock if the composed raid device (it may have metadata for a higher level device, like nested raid or LVM over RAID etc) e) for each raid members + spares mdadm fail device mdadm remove device f) mdadm stop g) mdadm zero_device h) wait for /dev/mdX to be released from the kernel I think we'd need to notice /dev/mdX is part of a container, and if so after tearing down the mdx, to then wipe the container? You mentioned the delete curtin does isn't sufficient; If you have the curtin install.log with the failure, that'll help sort this bit out. https://www.intel.com/content/dam/support/us/en/documents/memory-and- storage/ssd-software/Linux_VROC_6-0_User_Guide.pdf That has more details. specifically mdadm -vS which stops volumes and if we support the multi-volume setup, then removing sub-arrays is more complicated > 4) Yes, and even the EFI partition can be on RAID. Btw, it doesn't really > works with the legacy BIOS, mdadm doesn't find the raid support on our > servers without EFI. I'm not sure if it's supposed to work with legacy BIOS, > or it's just a firmware issue. EFI really shouldn't be on RAID in that EFI is VFAT and I don't believe there are EFI drivers for mdadm, maybe Intel provides a VROC/mdadm EFI driver, do you know? For legacy, the open question is whether grub2 can read an mdadm with metadata imsm metadata... sounds like no. > 5) That need to be checked, as the cleaning info comes from there. > > Yeah, I fear it cannot be automatically tested without actual hardware. =( > > > Maybe it would be possible to abstract out the container, like: > - type: raid > metadata: imsm > container: /dev/md/imsm0 > name: imsm0 > devices: > - /dev/nvme0n1p1 > - /dev/nvme1n1p1 > > - type: raid > devices: > - /dev/md/imsm0 # but need to get the number of real devices for -n We can do that by either repeating the values; it's a shame that the mdadm implementation doesn't just take the container name and look up the disks. It seems like a big oversite to require repeating the devices names since as you mention they have to keep the same devices and numbers. > name: mirror0 > level: 1 Given that to create multiple volumes from the container, we'll need to use two entries, the raid volumes will need a size. - type: raid id: disk_raid_container0 metadata: imsm name: /dev/md/imsm0 devices: - /dev/nvme0n1p1 - /dev/nvme1n1p1 - type: raid: id: disk_raid0_c0_vol0 level: 1 name: /dev/md/mirror0 container: disk_raid_container0 size_kb: - type: raid: id: disk_raid0_c0_vol1 level: 0 name: /dev/md/striped0 container: disk_raid_container0 size_kb: -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
Cool! The first step is to understand what additional metadata is needed to construct an VROC device (this container). The existing storage config yaml for raid looks like this: https://curtin.readthedocs.io/en/latest/topics/storage.html#raid-command It looks like we need to have a default name for the container, you use /dev/md/insm0, I see other places use /dev/md/insm. Thoughts? Maybe this metadata syntax: - type: raid metadata: imsm container: /dev/md/ism0 name: mirror0 devices: - /dev/nvme0n1p1 - /dev/nvme1n1p1 level: 1 Which would run two commands: mdadm --create /dev/md/ism0 --metadata=imsm --raid-devices=2 /dev/nvme0n1p1 /dev/nvme1n1p1 mdadm --create /dev/md0 ... /dev/md/ism0 --name=mirror0 Open questions/TODO: 1) does the second mdadm command need --metadata=imsm, --raid-devices=2 ? 2) it appears that the second command does require specifying raid-level, but what about devices? 3) how does one remove imsm containers? Curtin attempts to remove existing storage layers on top of physical disks so they can be re-used in other layered storages 4) can we boot to root on imsm devices? if so, what grub and initrd changes may be needed? 5) once created, we need to adjust mdadm examine/detail output parsing in curtin/block/mdadm.py; I believe the output of imsm device details is broken. Lastly; it's going to be hard to support this since we cannot emulate such a device in QEMU; curtin heavily relies on virtualization to test for regressions. Typically we'd construct an QEMU vm with storage that looks like an Intel VROC raid and make sure we can create/destroy these things automatically. ** Changed in: curtin (Ubuntu) Importance: Undecided => Wishlist ** Changed in: curtin (Ubuntu) Status: Incomplete => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
Thanks. I won't mark this as a duplicate yet without further confirmation. Would you be interested in working on the feature? I don't have access to any Intel VROC devices which makes it difficult to implement. I don't believe QEMU or other virt platforms emulate Intel VROC devices; one needs to have access to real hardware for testing. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1893661] Re: Support for Intel VROC (Virtual RAID On CPU)
Hello, Thanks for filing the bug. Do you know if VROC is different feature than RSTe? There's an existing feature request for that here: https://bugs.launchpad.net/curtin/+bug/1790055 ** Changed in: curtin (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1893661 Title: Support for Intel VROC (Virtual RAID On CPU) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1893661/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876258] Re: ubuntu 20.04 pxe installation fails with no such file or directory /dev/disk/by-id exception
Curtin is collecting by-id links to disk to populate grub2 's debconf value: grub-pc grub-pc/install_devices It's not clear to me (yet) if providing non persistent values (like /dev/sda) will work with /usr/lib/grub/grub-multi-install Even if it accepts non-by-id values; it's possible that on subsequent boots (with additional disks present) that /dev/sda might not point to the disk that has grub. So we can change this from an error by falling back on the path value, but it's just not safe. Disks need persistent attributes (like serial) for persistent links to work as expected. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876258 Title: ubuntu 20.04 pxe installation fails with no such file or directory /dev/disk/by-id exception To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1876258/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891608] Re: openvswitch-switch isn't installed if netplan schema defines an ovs bridge
This bug was fixed in the package curtin - 20.1-20-g1304d3ea-0ubuntu1 --- curtin (20.1-20-g1304d3ea-0ubuntu1) groovy; urgency=medium * New upstream snapshot. - curthooks: uefi_find_grub_device_ids handle type:mount without path (LP: #1892242) - netplan openvswitch yaml changed (LP: #1891608) - tools/curtainer: do not wait for snapd.seeded.service - tools/curtainer: enable using ubuntu-minimal images - vmtests: add Groovy [Paride Legovini] - Drop the Eoan vmtests (EOL) [Paride Legovini] - tools: rename remove-vmtest-release to vmtest-remove-release - Snooze the tests failing because of LP: #1861941 for two more months [Paride Legovini] - LP: #1671951 is Fix Released => Drop the PPA [Paride Legovini] - swaps: handle swapfiles on btrfs (LP: #1884161) - curtainer: fail is masking of zfs-mount or zfs-share fails [Paride Legovini] - multipath: handle multipath nvme name fields correctly (LP: #1878041) - curtainer: mask the zfs-mount and zfs-share services [Paride Legovini] - tools/jenkins-runner: shuffle test-cases to randomize load [Paride Legovini] - Add Trusty/UEFI/HWE-X vmtest, drop realpath add, drop shell code - LP: #1881977 - Install realpath on Trusty UEFI. [Lee Trager] - vmtests: fix PreservePartitionWipeVg storage config - Fix mdraid name creates broken configuration [James Falcon] (LP: #1803933) - vmtests: update skiptests - vmtest: allow installed centos images to reboot (LP: #1881011) -- Ryan Harper Thu, 20 Aug 2020 09:10:33 -0500 ** Changed in: curtin (Ubuntu Groovy) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891608 Title: openvswitch-switch isn't installed if netplan schema defines an ovs bridge To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1891608/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891608] Re: openvswitch-switch isn't installed if netplan schema defines an ovs bridge
Tomorrow's Curtin daily:ppa should have this (and as you can see, other fixes) https://launchpad.net/~curtin-dev/+archive/ubuntu/daily -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891608 Title: openvswitch-switch isn't installed if netplan schema defines an ovs bridge To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1891608/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891608] Re: openvswitch-switch isn't installed if netplan schema defines an ovs bridge
** Also affects: curtin (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Bionic) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Groovy) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Focal) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891608 Title: openvswitch-switch isn't installed if netplan schema defines an ovs bridge To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1891608/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891608] Re: openvswitch-switch isn't installed if netplan schema defines an ovs bridge
** Also affects: curtin (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891608 Title: openvswitch-switch isn't installed if netplan schema defines an ovs bridge To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1891608/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1892242] Re: Curtin doesn't handle type:mount entries without 'path' element
Tomorrows Curtin daily:ppa should have this (and as you can see, other fixes) https://launchpad.net/~curtin-dev/+archive/ubuntu/daily -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1892242 Title: Curtin doesn't handle type:mount entries without 'path' element To manage notifications about this bug go to: https://bugs.launchpad.net/curtin/+bug/1892242/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working
Updated image is available: % image-status maas3 | grep groovy groovy amd64/ga-20.10/generic 20200819.1 squashfs groovy amd64/ga-20.10/lowlatency20200819.1 squashfs GroovyTestBasic on amd64 passes. Thu, 20 Aug 2020 09:40:51 -0500: vmtest start: nosetests3 -vv --nologcapture tests/vmtests/test_basic.py:GroovyTestBasic nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$'] fatal: No names found, cannot describe anything. fatal: No names found, cannot describe anything. 2020-08-20 09:40:51,547 - vmtests - INFO - Logfile: /home/rharper/work/git/curtin/output/debug.log . Working dir: /home/rharper/work/git/curtin/output 2020-08-20 09:40:51,550 - GroovyTestBasic - INFO - Logfile: /home/rharper/work/git/curtin/output/debug.log . Working dir: /home/rharper/work/git/curtin/output 2020-08-20 09:40:51,553 - GroovyTestBasic - INFO - Starting setup for testclass: GroovyTestBasic (ubuntu/groovy -> ubuntu/groovy) 2020-08-20 09:40:51,619 - GroovyTestBasic - INFO - Using tempdir: /home/rharper/work/git/curtin/output/GroovyTestBasic 2020-08-20 09:40:51,620 - GroovyTestBasic - INFO - Loading testcase config file: examples/tests/basic.yaml => groovy/amd64/20200819.1/ga-20.10/generic/boot-initrd [86118105] .^[[5;2~.. => groovy/amd64/20200819.1/ga-20.10/generic/boot-kernel [11662080] ... => groovy/amd64/20200819.1/squashfs [377270272] ... 2020-08-20 09:42:12,196 - GroovyTestBasic - INFO - Publishing ephemeral image as --publish=/srv/images/groovy/amd64/20200819.1/squashfs:root/squashfs 2020-08-20 09:42:12,196 - GroovyTestBasic - INFO - Curtin install source URI: cp:///media/root-ro 2020-08-20 09:42:12,207 - GroovyTestBasic - INFO - Loading testcase config file: examples/tests/basic.yaml 2020-08-20 09:42:12,214 - GroovyTestBasic - INFO - disk_serials: ['serial=disk-a', 'serial=disk-b', 'serial=disk-c', 'serial=disk-d'] 2020-08-20 09:42:12,214 - GroovyTestBasic - INFO - nvme_serials: [] 2020-08-20 09:42:12,214 - GroovyTestBasic - INFO - nvme disks: [] 2020-08-20 09:42:12,231 - GroovyTestBasic - INFO - Using root_disk: [] 2020-08-20 09:42:12,231 - GroovyTestBasic - INFO - Running curtin installer: /home/rharper/work/git/curtin/output/GroovyTestBasic/logs/install-serial.log 2020-08-20 09:44:42,576 - GroovyTestBasic - INFO - GroovyTestBasic[install]: boot took 150.34 seconds. returned True 2020-08-20 09:44:42,896 - GroovyTestBasic - INFO - Install OK 2020-08-20 09:44:42,897 - GroovyTestBasic - INFO - Booting target image: /home/rharper/work/git/curtin/output/GroovyTestBasic/logs/boot-serial.log 2020-08-20 09:44:54,320 - GroovyTestBasic - INFO - GroovyTestBasic[first_boot]: boot took 11.42 seconds. returned True 2020-08-20 09:44:54,367 - GroovyTestBasic - INFO - GroovyTestBasic: setUpClass finished. took 242.81 seconds. Running testcases. get_test_files (vmtests.test_basic.GroovyTestBasic) ... ok test_clear_holders_ran (vmtests.test_basic.GroovyTestBasic) ... ok test_curtin_install_version (vmtests.test_basic.GroovyTestBasic) ... ok test_dname (vmtests.test_basic.GroovyTestBasic) ... ok test_dname_rules (vmtests.test_basic.GroovyTestBasic) ... ok test_fstab (vmtests.test_basic.GroovyTestBasic) ... ok test_installed_correct_kernel_package (vmtests.test_basic.GroovyTestBasic) ... ok test_interfacesd_eth0_removed (vmtests.test_basic.GroovyTestBasic) ... ok test_kernel_img_conf (vmtests.test_basic.GroovyTestBasic) ... SKIP: LP: #1847257 kernel-img.conf not needed in eoan and newer test_output_files_exist (vmtests.test_basic.GroovyTestBasic) ... ok test_partition_is_prep (vmtests.test_basic.GroovyTestBasic) ... ok test_partition_is_zero (vmtests.test_basic.GroovyTestBasic) ... ok test_partition_numbers (vmtests.test_basic.GroovyTestBasic) ... ok test_proxy_set (vmtests.test_basic.GroovyTestBasic) ... ok test_ptable (vmtests.test_basic.GroovyTestBasic) ... ok test_reporting_data (vmtests.test_basic.GroovyTestBasic) ... ok test_swaps_used (vmtests.test_basic.GroovyTestBasic) ... ok test_whole_disk_uuid (vmtests.test_basic.GroovyTestBasic) ... ok -- Ran 18 tests in 246.692s OK (SKIP=1) Thu, 20 Aug 2020 09:44:58 -0500: vmtest end [0] in 248s -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1890803 Title: Groovy amd64 / arm64 / PowerPC deployment seems not working To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working
Once a new maas image is published with the updated initramfs, I can verify; As of today, I still only see 20200814 for groovy % date Wed 19 Aug 2020 11:03:04 AM CDT % image-status maas3 | grep groovy groovy amd64/ga-20.10/generic 20200814squashfs groovy amd64/ga-20.10/lowlatency20200814squashfs -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1890803 Title: Groovy amd64 / arm64 / PowerPC deployment seems not working To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working
https://code.launchpad.net/~raharper/cloud-initramfs-tools/+git/cloud- initramfs-tools/+merge/389422 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1890803 Title: Groovy amd64 / arm64 / PowerPC deployment seems not working To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1890803] Re: Groovy amd64 / arm64 / PowerPC deployment seems not working
It turns out the 20200814 squashfs from maas *now* has an /etc/fstab in it ... So after overlayroot runs, we have an fstab like this: root@(none):/# cat /etc/fstab LABEL=cloudimg-rootfs /ext4 defaults0 0 # # This fstab is for overlayroot. The real one can be found at # /media/root-ro/etc/fstab # The original entry for '/' and other mounts have been updated to be placed # under /media/root-ro. # To permanently modify this (or any other file), you should change-root into # a writable view of the underlying filesystem using: # sudo overlayroot-chroot # #LABEL=cloudimg-rootfs /media/root-ro/ ext4 ro,defaults,noauto 0 0 /media/root-ro/ / overlay lowerdir=/media/root-ro/,upperdir=/media/root-rw/overlay/,workdir=/media/root-rw/overlay-workdir/_ 0 0 Overlayroot needs to handle with and without fstab entry so we'll need to fix that; However it's not clear why there is now an fstab entry in squashfs. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1890803 Title: Groovy amd64 / arm64 / PowerPC deployment seems not working To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1890803/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
@Rafael It's in both places: https://github.com/koverstreet/bcache-tools/pull/1 I can update the PR there as well; though I don't think upstream cares as Kent's working on bcachefs instead of bcache AFAICT. > I see that you are trying to come up with something nor relying in the kernel fix (to throw the variables again after pivot root, etc..) I think after the systemd discussion upstream: https://github.com/systemd/systemd/pull/16317 The kernel uevent for CACHED_UUID isn't the right way to go long term; all of the information we want is present in the superblock of the devices as they are probed; If we have a 69-bcache.rules with a helper that uses bcache-super-show; we could drop the Ubuntu sauce patch we have; it's just noise (and now it's somewhat unreliable, this bug shows that). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
@Rafael For bcache-tools changes, bcache-export-cached-uuid needs the full path to bcache-super-show, as PATH is not exported when running from udev. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886107] Re: centos8 rpm build with odd runtime deps
I think we want two flags; keep the default behavior of all deps output: and then --build-requires # print the build package requirements --requires # print the runtime package requirements Then brpm needs to be updated to call read-deps with those two flags, and adjust its build-requires and requires accordingly. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886107 Title: centos8 rpm build with odd runtime deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1886107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886080] Re: lxd cloud_tests fail to export images on latest/stable
** Changed in: cloud-init (Ubuntu) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886080 Title: lxd cloud_tests fail to export images on latest/stable To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1886080/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886531] Re: cloud-init status broken in groovy lxd containers
https://github.com/canonical/cloud-init/pull/484 ** Changed in: cloud-init Importance: Undecided => Medium ** Changed in: cloud-init Assignee: (unassigned) => Ryan Harper (raharper) ** Changed in: cloud-init Status: New => In Progress -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886531 Title: cloud-init status broken in groovy lxd containers To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1886531/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886107] [NEW] centos8 rpm build with odd runtime deps
Public bug reported: % ./tools/run-container centos/8 --package --artifacts=. ... % rpm -U cloud-init-20.2+89.g818da27a-1.el8.noarch.rpm Command: rpm -U /tmp/cloud-init-20.2+89.g818da27a-1.el8.noarch.rpm Exit code: 1 Reason: - Stdout: Stderr: error: Failed dependencies: python3-devel is needed by cloud-init-20.2+89.g818da27a-1.el8.noarch sudo is needed by cloud-init-20.2+89.g818da27a-1.el8.noarch python3-devel has to be a build-dep; sudo is require though. ** Affects: cloud-init (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1886107 Title: centos8 rpm build with odd runtime deps To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1886107/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
@ddstreet I'm not sure where upstream is going just yet. For Ubuntu; I think 1) Adjusting the bcache-tools patch to use the full path to bcache- super-show should change; 2) If we fix (1) then I think we can drop the systemd patch from a bug fixing perspective; on the openSUSE image I did testing on; the skip of the dev/disk/by-uuid symlink for backing devices does not *break* the /dev/bcache/by-uuid ; it was missing due to the updated 69-bcache.rules not working. 3) IMO, blkid should not report FS_UUID for bcache backing/cache devices since it's not a Filesystem; we could patch that in blkid directly instead; alternatively since older blkid's may or maynot emit FS_UUID for bcache backing/caching devices the patch to systemd we're caring will avoid creating a useless dev/disk/by-uuid symlink to those devices. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1886080] [NEW] lxd cloud_tests fail to export images on latest/stable
Public bug reported:
% tox -e citest -- run --verbose --os-name centos70 --preserve-data --data-dir
ctestout --test bugs/lp1885952-chrony
...
Command: ['lxc', 'image', 'export',
'b73b65fa3d3ce2b5aad9c22ac1c83575c15262e157db5651219318aad13cb94f',
'/tmp/cloud_test_util_9kido331']
Exit code: 1
Reason: -
Stdout:
Stderr: Error: open /var/lib/snapd/hostfs/tmp/cloud_test_util_9kido331: is a
directory
2020-07-02 17:16:24,585 -
/home/rharper/work/git/cloud-init/tests/cloud_tests/stage.py:run_stage:104
[ERROR]: traceback:
File "/home/rharper/work/git/cloud-init/tests/cloud_tests/stage.py", line 97,
in run_stage
(call_res, call_failed) = call()
File "/home/rharper/work/git/cloud-init/tests/cloud_tests/collect.py", line
169, in collect_image
with component as image:
File "/home/rharper/work/git/cloud-init/tests/cloud_tests/stage.py", line 27,
in __enter__
self.instance = self.get_func()
File
"/home/rharper/work/git/cloud-init/tests/cloud_tests/platforms/__init__.py",
line 22, in get_image
return platform.get_image(config)
File
"/home/rharper/work/git/cloud-init/tests/cloud_tests/platforms/lxd/platform.py",
line 38, in get_image
image.update_templates(self.config.get('template_overrides', {}),
File
"/home/rharper/work/git/cloud-init/tests/cloud_tests/platforms/lxd/image.py",
line 127, in update_templates
(metadata, rootfs) = self.export_image(export_dir)
File
"/home/rharper/work/git/cloud-init/tests/cloud_tests/platforms/lxd/image.py",
line 85, in export_image
subp.subp(['lxc', 'image', 'export', self.pylxd_image.fingerprint,
File "/home/rharper/work/git/cloud-init/cloudinit/subp.py", line 290, in subp
raise ProcessExecutionError(stdout=out, stderr=err,
1) lxc image exportis the new format; if
is a directly like platforms/lxd/image.py provides, it fails.
The export format now writes two file
% lxc image export myimage
myimage
myimage.root
Where myimage is a tar.xz of the metadata folder and myimage.root is the
squashfs root
(crispyboi) lxd % lxc image export --verbose
e37c4d8078f197d3eabd85a9bb5a87462db54b5d6729cff63d86bdd5c312ad40 myimage.tar.xz
Image exported successfully!
(crispyboi) lxd % ls -al
total 85272
drwxrwxr-x 2 rharper rharper4 Jul 2 12:20 .
drwxrwxrwt 49 rootroot 59 Jul 2 12:19 ..
-rw-rw-r-- 1 rharper rharper 844 Jul 2 12:20 myimage.tar.xz
-rw-rw-r-- 1 rharper rharper 87203840 Jul 2 12:20 myimage.tar.xz.root
(crispyboi) lxd % tar tJvf myimage.tar.xz
-rw-r--r-- root/root 686 2020-07-01 02:31 metadata.yaml
drwxr-xr-x root/root 0 2020-07-01 02:31 templates/
-rw-r--r-- root/root 189 2020-07-01 02:31 templates/hosts.tpl
-rw-r--r-- root/root 139 2020-07-01 02:31 templates/ifcfg-eth0.lxd.tpl
-rw-r--r-- root/root45 2020-07-01 02:31 templates/network.lxd.tpl
(crispyboi) lxd % file myimage.tar.xz.root
myimage.tar.xz.root: Squashfs filesystem, little endian, version 1024.0,
compressed, 1701289756295430144 bytes, -1455882240 inodes, blocksize: 4096
bytes, created: Thu Feb 7 01:07:42 2104
The following seems to make it work:
diff --git a/tests/cloud_tests/platforms/lxd/image.py
b/tests/cloud_tests/platforms/lxd/image.py
index 8934fb74..172f83ab 100644
--- a/tests/cloud_tests/platforms/lxd/image.py
+++ b/tests/cloud_tests/platforms/lxd/image.py
@@ -82,13 +82,11 @@ class LXDImage(Image):
@return_value: tuple of path to metadata tarball and rootfs tarball
"""
# pylxd's image export feature doesn't do split exports, so use cmdline
+outfile = output_dir + 'myimage.tar.xz'
subp.subp(['lxc', 'image', 'export', self.pylxd_image.fingerprint,
- output_dir], capture=True)
-tarballs = [p for p in os.listdir(output_dir) if p.endswith('tar.xz')]
-metadata = os.path.join(
-output_dir, next(p for p in tarballs if p.startswith('meta-')))
-rootfs = os.path.join(
-output_dir, next(p for p in tarballs if not p.startswith('meta-')))
+ outfile], capture=True)
+metadata = outfile
+rootfs = outfile + '.root'
return (metadata, rootfs)
** Affects: cloud-init (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1886080
Title:
lxd cloud_tests fail to export images on latest/stable
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1886080/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
On Tue, Jun 30, 2020 at 6:35 AM Balint Reczey <1861...@bugs.launchpad.net> wrote: > @raharper I've forwarded the systemd fix for you with minimal tidying of > the commit message https://github.com/systemd/systemd/pull/16317 Thanks! > > -- > You received this bug notification because you are subscribed to the bug > report. > https://bugs.launchpad.net/bugs/1861941 > > Title: > bcache by-uuid links disappear after mounting bcache0 > > To manage notifications about this bug go to: > > https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions > -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878890] Re: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning
> @Ryan, > > Since the installer refuses to install, there are no crash logs that I can > provide you with. If you try to install 20.04 onto a disk with the mentioned > layout (the "Custom layout" option), you should find that you are unable to > choose sda1 and sda3 as destinations for /boot and /. Don't you have a system > where you can emulate this (that would be very handy)? Thank you for the repro steps. Booting the curtin live focal iso, I see that the curtin version included with subiquity is 19.3-67-g145e49393 And this fix for this issue is in curtin 19.3-76-gcb60d8a7. The fix has landed in *curtin* but we still need subiquity to pull in this fix (subiquity bundles curtin into the snap). > > Is the installer log located in the same folder as the crash logs > (/run/casper or something like that)? It should be in /var/log/installer/* and /var/log/curtin/* (if it gets that far, which in your case it doesnt) > > A rant for another time is that the 20.04 installer system has never stored > the install and crash logs on installation USB stick. In order to retrieve the > logs I've had to use a second USB stick and manually copy files over to it. > Crashes typically are uploaded to the crash service; however in the case where it doesn't actually crash; there's not a way to "report an issue"; that I know of, so please do file a bug for that so we can track getting a fix for that. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878890 Title: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1878890/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878890] Re: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning
@Kai Can you provide the installer crash or logs? "refuses to use /dev/sda1 as boot partition and /dev/sda3 as root partition" I wonder if recent changes in subiquity around ESPs may have something to do with this. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878890 Title: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1878890/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
** Changed in: cloud-init (Ubuntu) Importance: Undecided => High ** Changed in: cloud-init (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
> Ryan, > > Thank you so much for taking the time to go over the constraints and > competing goals. It's always helpful to know what constituencies are > involved. Sure. > > > That said, I do think that cloud-init can make it easier to enable ipv6 > > where > > it's desired (but without having to write your own network config). We're > > looking into supporting a kernel parameter or cloud-config option to enable > > the default config to block on ipv6 (note, we cannot use user-data to > > indicate > > this preference as user-data is processed after networking is up as > > user-data > > can be fetched via network URL). > > I think this would be a good compromise to get things moving. In my case of > downloading the cloud image and specifying NoCloudNet via the SMBIOS serial > number option, another key might be the simplest path. Something like: > > n=4 : configure IPv4 only (default now) > n=6 : configure IPv6 only > n=64 : configure both stacks > > I might then pass in: > ds=nocloud-net;n=6;h=cloudtest;i=iid-local01;s=http://gemini.home.nivex.net/cidata/ I like this a lot. This example would only apply to NoCloud datasource shorthand, which is fine, but we should expand this into a general form where users can control the policy for any datasource. I suspect we might bike-shed a bit on the long and short form of the config. We internally refer to 'fallback' network config, and to express what the current default configuration in long form. I've started a hackmd document. https://hackmd.io/hoAMkuQ8S_uj62ot3St9fw > > Of course it's easy for me to throw out such a suggestion since I don't know > how much is involved in the codebase. Is this something that might be doable > in the near future? Definitely! Would you be interested in contributing? We can help shepard the work along the way. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1881003] Re: sru curtin 2020-05-27 - 20.1-0ubuntu1
** Description changed: == Begin SRU Template == [Impact] This release sports both bug-fixes and new features and we would like to make sure all of our supported customers have access to these improvements. The notable ones are: -* + * See the changelog entry below for a full list of changes and bugs. [Test Case] The following development and SRU process was followed: https://wiki.ubuntu.com/CurtinUpdates Curtin now contains an extensive integration test suite that is ran using the SRU package for each releases. These suite has documentation here: https://curtin.readthedocs.io/en/latest/topics/integration-testing.html In order to avoid regression to existing MAAS product, the MAAS team will run their continuous integration test against the curtin that is in -proposed. A successful run will be required before the proposed curtin can be let into -updates. The curtin team will be in charge of attaching the artifacts and console output of the appropriate run to the bug. Curtin team members will not mark ‘verification-done’ until this has happened. [Regression Potential] In order to mitigate the regression potential, the results of the aforementioned integration tests are attached to this bug. [Discussion] == End SRU Template == - - + * New upstream snapshot. (LP: #1881003) + - vmtests: update skiptests + - vmtest: allow installed centos images to reboot (LP: #1881011) + - Release 20.1 (LP: #1880741) + - Handle multiple separators which were found in TestAllindata vmtest + - verify_ptable_flag: dos primary partitions use ptable_uuid map for flag + (LP: #1878890) + - net_meta: add disabled mode to skip writing any network config + [Lucas Moura] + - vmtest: trigger guest panic to fail fast + - Replace grub-shell-helper with install_grub command + - vmtest-sync-images: update the URL of the maas streams [Paride Legovini] + - Replace references to old team manager with new team manager + [James Falcon] + - tox: pin flake8 to version and add a tip-flake8 environment + [Paride Legovini] + - Fix flake8 E741 warning [Lucas Moura] + - Makefile: make adjustments to call lint/style tools via python module + - block-discover: ignore invalid id_serial/id_wwn values (LP: #1876848) + - Fix handing of reusing msdos partitions and flags (LP: #1875903) + - block.detect_multipath: ignore fake "devices" from /proc/mounts + [Michael Hudson-Doyle] (LP: #1876626) + - udev: use shlex.quote when shlex.split errors on shell-escape chars + (LP: #1875085) + - lvm: don't use vgscan --mknodes + - vmtest: rsync don't cross filesystem boundaries when copying + (LP: #1873909) + - vmtest: basic/basic_scsi adjust collect/tests for unstable device names + (LP: #1874100) + - Add unittests for partition_handler calc_[dm]_part_info and kpartx paths + - multipath: attempt to enforce /dev/mapper/mpath files are symlinks + - block-meta: device mapper partitions may be block devices not links + - Default to dm_name being id if empty earlier in dm_crypt_handler() + [Łukasz 'sil2100' Zemczak] (LP: #1874243) + - storage: correct declared schema draft version for storage schema + - test_clear_holders: add missing zfs mock + - Mock out zfs_supported to prevent attempting to load kernel modules + - block-meta: skip wipe device paths if not present (LP: #1869075) + - unittest: do not allow util.subp by default (LP: #1873913) + - curthooks: support multiple ESP on UEFI bootable systems + - block-discover: handle missing multipath 'path' data, use DM_NAME + (LP: #1873728) + - lvm-over-multipath: handle lookups of multipath members (LP: #1869075) + - block-meta: don't filter preserve=true devices, select by wipe + (LP: #1837214) + - vmtest: basic use dname to lookup disk with multiple partitions + - block-meta: Don't check the ptable type of a disk with no ptable + - curthooks: always use ChrootableTarget.subp when calling efibootmgr + - storage: enable and use multipath during storage configuration + (LP: #1869075) + - block-discover: detect nvme multipath devices (LP: #1868109) + - clear-holders: Tolerate vgchange errors during discovery (LP: #1870037) + - block-meta: handle preserve with vtoc ptable (LP: #1871158) + - vmtest: use -partition file for TestReuseRAIDMemberPartition class + - format: extra_options should be a list type + - tox: add pyflakes to the default tox run [Paride Legovini] + - storage_config: Add 'extra_options' parameter to allow custom mkfs + (LP: #1869069) + - Add support for installing Ubuntu Core 20 images + - tox.ini: Fix issues with newer tox on focal + - vmtest: Fix test_basic.py to run on s390x (LP: #183) + - vmtest: use util.load_file for loading collect files + - block-meta: refactor storage_config preserve and wipe settings + (LP: #1837214) + - block-discover: skip 'multipath' key in blockdevice if mpath name is None + - tox: all py27 environments should use the base py27 deps + - uefi: refactor
[Bug 1881030] [NEW] centos70 uefi shim path does not include mount path when installed
Public bug reported: Not quite sure if what to do about this one; it's possible the "initial setup" steps inject one path that the install path doesn; but the result is that the "duplicate" entries aren't considered similar as the path (and the name) don't match. There's nothing *fatal* about this; it's OK to have duplicate entries but if we could remove the duplicate we should. For now, this bug can be used to skip the Cento70 version of this test until we can sort out how best to move forward. ** Affects: curtin (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1881030 Title: centos70 uefi shim path does not include mount path when installed To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1881030/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1881011] [NEW] vmtest: no-reboot prevents installed centos images from reporting success
Public bug reported: Centos vmtests install fine, the first boot phase fails as Centos requires an selinux relabling process which reboots after relabeling. The --no-reboot flag prevents this and we fail the test. *** Warning -- SELinux targeted policy relabel is required. *** Relabeling could take a very long time, depending on file vmtest should not specify the --no-reboot flag for the first-boot phase if on centos. ** Affects: curtin (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1881011 Title: vmtest: no-reboot prevents installed centos images from reporting success To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1881011/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1881003] [NEW] sru curtin 2020-05-27 - 20.1-0ubuntu1
Public bug reported: == Begin SRU Template == [Impact] This release sports both bug-fixes and new features and we would like to make sure all of our supported customers have access to these improvements. The notable ones are: * See the changelog entry below for a full list of changes and bugs. [Test Case] The following development and SRU process was followed: https://wiki.ubuntu.com/CurtinUpdates Curtin now contains an extensive integration test suite that is ran using the SRU package for each releases. These suite has documentation here: https://curtin.readthedocs.io/en/latest/topics/integration-testing.html In order to avoid regression to existing MAAS product, the MAAS team will run their continuous integration test against the curtin that is in -proposed. A successful run will be required before the proposed curtin can be let into -updates. The curtin team will be in charge of attaching the artifacts and console output of the appropriate run to the bug. Curtin team members will not mark ‘verification-done’ until this has happened. [Regression Potential] In order to mitigate the regression potential, the results of the aforementioned integration tests are attached to this bug. [Discussion] == End SRU Template == ** Affects: curtin (Ubuntu) Importance: Undecided Status: New ** Affects: curtin (Ubuntu Xenial) Importance: Undecided Status: New ** Affects: curtin (Ubuntu Bionic) Importance: Undecided Status: New ** Affects: curtin (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Focal) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Xenial) Importance: Undecided Status: New ** Also affects: curtin (Ubuntu Bionic) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1881003 Title: sru curtin 2020-05-27 - 20.1-0ubuntu1 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1881003/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
Hi Kevin, > In the year 2020, IPv6 should be a first class citizen and I should not need > to provide additional config for it to work right. The issue here is not > that I can't find a workaround to make my scenarios work. The issue here is > that I should not need any such workarounds on a modern network. Thanks. I understand, and in general, agree with the goal. Our challenge is how to balance default configurations with competing goals. We can enable lots of default features for maximum compatibility; however there is almost always a trade-off. The trade off for IPv6 is significant delay during boot if IPv6 is not present. Cloud-init blocks during boot to ensure that networking is configured (as expected) before continuing to boot; later stages of boot can expect that networking is available for fetching data from the network. In the case of IPv6, and *blocking* for IPv6 during boot there is a very high cost in networks where IPv6 is not available. The Ubuntu kernel already accepts IPV6 Router Advertisements by default, but asynchronously from the systems network configuration. As mentioned, we can enable blocking on RAs by default but there is a very large boot time delay for networks without RAs. As an example, I've two instance differing only in one where I've enabled blocking for RAs and the boot time is 12 seconds slower due to waiting on RAs. # default fallback networking (dhcp4 only) root@g3:~# systemd-analyze Startup finished in 8.132s (userspace) graphical.target reached after 7.603s in userspace # dhcp4 and accept-ra: true root@g4:~# systemd-analyze Startup finished in 19.988s (userspace) graphical.target reached after 19.472s in userspace # using systemd-analyze critical-chain we can see where the time went root@g3:~# systemd-analyze critical-chain The time when unit became active or started is printed after the "@" character. The time the unit took to start is printed after the "+" character. graphical.target @7.603s └─multi-user.target @7.602s └─rsyslog.service @2.524s +5.074s └─basic.target @2.491s └─sockets.target @2.491s └─snapd.socket @2.489s +2ms └─sysinit.target @2.485s └─cloud-init.service @1.853s +631ms └─systemd-networkd-wait-online.service @1.608s +243ms └─systemd-networkd.service @1.326s +279ms └─network-pre.target @1.325s └─cloud-init-local.service @747ms +578ms └─systemd-journald.socket @165ms └─system.slice @162ms └─-.slice @162ms root@g4:~# systemd-analyze Startup finished in 19.988s (userspace) graphical.target reached after 19.472s in userspace root@g4:~# systemd-analyze critical-chain The time when unit became active or started is printed after the "@" character. The time the unit took to start is printed after the "+" character. graphical.target @19.472s └─multi-user.target @19.472s └─rsyslog.service @14.378s +5.092s └─basic.target @14.338s └─sockets.target @14.338s └─snapd.socket @14.333s +4ms └─sysinit.target @14.329s └─cloud-init.service @13.686s +642ms └─systemd-networkd-wait-online.service @1.672s +12.010s └─systemd-networkd.service @1.376s +290ms └─network-pre.target @1.375s └─cloud-init-local.service @781ms +593ms └─systemd-journald.socket @159ms └─system.slice @156ms └─-.slice @156ms And with debugging on we can see where networkd is sending Router Solicitations and waiting for a response. [205653.809964] g4 systemd[1]: Starting Network Service... ... [205654.706539] g4 systemd-networkd[191]: NDISC: Sent Router Solicitation, next solicitation in 4s [205658.783325] g4 systemd-networkd[191]: NDISC: Sent Router Solicitation, next solicitation in 7s [205659.231119] g4 systemd-networkd[191]: rtnl: received non-static neighbor, ignoring. [205666.110936] g4 systemd-networkd[191]: NDISC: No RA received before link confirmation timeout [205666.112556] g4 systemd-networkd[191]: NDISC: Invoking callback for 'timeout' event. [205666.112822] g4 systemd-networkd[191]: eth0: State changed: configuring -> configured [205666.113225] g4 systemd-networkd-wait-online[192]: managing: eth0 [205666.116688] g4 systemd[1]: Finished Wait for Network to be Configured. If we were to change the default to block for IPv6, and every cloud image boot were to block 12 seconds waiting for an RA, I'm positive the opposite bug would be filed requesting that we don't enable ipv6 when it's not widely deployed. On clouds where ipv6 is available, those platforms provide cloud-init with networking configuration indicating that cloud-init should turn on ipv6 and expect a response. This prevents instances from waiting for ipv6 responses which will never come and delaying boot. That said, I do think that cloud-init
[Bug 1878890] Re: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning
@Michael I'll sync with mwhudson, subiquity will need to pull in the fix and release an update to edge, once that's done you can select the Update Installer option. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878890 Title: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning To manage notifications about this bug go to: https://bugs.launchpad.net/subiquity/+bug/1878890/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
systemd debdiff with a fix to skip creating /dev/disk/by-uuid for bcache backing, caching devices. ** Patch added: "lp1861941-skip-bcache-links.debdiff" https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+attachment/5375730/+files/lp1861941-skip-bcache-links.debdiff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
Tarball of a source package with a fix for this issue: bcache-tools_1.0.8.orig.tar.gz bcache-tools_1.0.8-4ubuntu1_amd64.build bcache-tools_1.0.8-4ubuntu1_amd64.buildinfo bcache-tools_1.0.8-4ubuntu1_amd64.changes bcache-tools_1.0.8-4ubuntu1_amd64.deb bcache-tools_1.0.8-4ubuntu1.debian.tar.xz bcache-tools_1.0.8-4ubuntu1.dsc bcache-tools_1.0.8-4ubuntu1_source.build bcache-tools_1.0.8-4ubuntu1_source.buildinfo bcache-tools_1.0.8-4ubuntu1_source.changes bcache-tools-dbgsym_1.0.8-4ubuntu1_amd64.ddeb bcache-tools-debdiff-1.0.8-4_to_1.0.8-4ubuntu1 ** Attachment added: "bcache-tools-fix-lp1861941.tar.gz" https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1861941/+attachment/5375721/+files/bcache-tools-fix-lp1861941.tar.gz -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
debdiff of the changes ** Attachment added: "bcache-tools-debdiff-1.0.8-4_to_1.0.8-4ubuntu1" https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1861941/+attachment/5375722/+files/bcache-tools-debdiff-1.0.8-4_to_1.0.8-4ubuntu1 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
Updated test to be a bit more resilient. ** Attachment added: "test-bcache-byuuid-links-fixed.sh" https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1861941/+attachment/5375723/+files/test-bcache-byuuid-links-fixed.sh -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1827929] Re: curtin should support making installed systems BIOS+UEFI hybrid bootable
Lastly, much of the focus on DOS partition tables comes from subiquity's re-use existing partitions; it's still extremely common to have a dos partition table rather than gpt. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1827929 Title: curtin should support making installed systems BIOS+UEFI hybrid bootable To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1827929/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1827929] Re: curtin should support making installed systems BIOS+UEFI hybrid bootable
> On Fri, May 22, 2020 at 04:09:51PM -0000, Ryan Harper wrote:
> > We will need two formats for the hybrid mode, ptable: gpt and ptable:
> > dos
>
> I don't see why this should be the case. Are we not emitting GPT everywhere
> today, even when the system is booting in BIOS mode? Late-model BIOSes do
> support booting from GPT; I would think at this point anything that doesn't
> support GPT is too old to be of practical concern.
>
> So a GPT table with a bios_grub partition and an EFI System Partition should
> suffice.
Curtin has to deal with either construct; The design of the storage config is
from MAAS, Subiquity, user-defined autoconf install, Field Engineer creating a
one-off, etc. What I'm saying here is that curtin should validate that
one can create a hybrid partition layout with either GPT and MSDOS. Note,
I first saw such a dos-based hybrid looking at the partition layout of our
own installer iso. =)
"/dev/sdc": {
"ID_FS_LABEL": "Ubuntu-Server_18.04.3_LTS_amd64",
"ID_FS_UUID_ENC": "2019-08-05-20-00-00-00",
"ID_FS_LABEL_ENC": "Ubuntu-Server\\x2018.04.3\\x20LTS\\x20amd64",
"ID_BUS": "usb",
"ID_FS_BOOT_SYSTEM_ID": "EL\\x20TORITO\\x20SPECIFICATION",
"ID_FS_TYPE": "iso9660",
"DEVNAME": "/dev/sdc",
"ID_PART_TABLE_TYPE": "dos",
"partitiontable": {
"label": "dos",
"id": "0x2f10bd40",
"unit": "sectors",
"partitions": [
{
"bootable": true,
"size": 1736704,
"type": "0",
"node": "/dev/sdc1",
"start": 0
},
{
"size": 4928,
"type": "ef",
"node": "/dev/sdc2",
"start": 1295064
}
],
"device": "/dev/sdc"
},
},
>
> > For gpt, the config from comment #1 applies, we'll have both bios_grub
> > and an efi partition (possibly a secondary for resilient boot).
>
> Resilient boot is only applicable in the case of multiple disks; and then
Yes
> each disk in the root set should have exactly one ESP and exactly one
> bios_grub partition.
Yes. We currently are testing UEFI with GPT layout only; with hybrid we
need to test that resilient boot works with the MSDOS hybrid layout as well.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1827929
Title:
curtin should support making installed systems BIOS+UEFI hybrid
bootable
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1827929/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
OK.
I've reviewed the kernel code, and there are no unexpected changes w.r.t
the CACHED_UUID change event. So I don't think we will need any kernel
changes which is good.
With the small change to the 60-persistent-storage.rules to not attempt
to create a /dev/disk/by-uuid symlink for the backing device; instead we
want to only create a /dev/bcache/by-uuid symlink to the bcacheN device
(that is associated with the backing device).
# by-label/by-uuid links (filesystem metadata), skip bcache backing,caching
devices, handled in 69-bcache.rules
ENV{ID_FS_TYPE}=="bcache", GOTO="skip_fs_uuid_enc"
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*",
SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_LABEL_ENC}=="?*",
SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"
LABEL="skip_fs_uuid_enc"
And then with my previously proposed changed to 69-bcache.rules in place
https://github.com/g2p/bcache-tools/pull/29/files
The test-case works just fine.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1861941
Title:
bcache by-uuid links disappear after mounting bcache0
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1827929] Re: curtin should support making installed systems BIOS+UEFI hybrid bootable
We will need two formats for the hybrid mode, ptable: gpt and ptable: dos For gpt, the config from comment #1 applies, we'll have both bios_grub and an efi partition (possibly a secondary for resilient boot). For dos, we'll have: config: - id: main_disk type: disk ptable: dos serial: disk-a name: main_disk wipe: superblock - device: main_disk id: main_disk_part1 number: 1 size: 10G type: partition grub_device: true flag: boot - device: main_disk id: main_disk_part2 number: 2 size: 512M type: partition grub_device: true flag: esp wipe: superblock Curtin currently uses 'boot' to mean toggle the DOS boot flag so we'll need to do something to trigger marking this. I've suggested 'esp' here to map to the 0xef code and we'll need to see if parted will let us do this, or use sfdisk or something else to directly toggle the type. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1827929 Title: curtin should support making installed systems BIOS+UEFI hybrid bootable To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1827929/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
Digging deeper and walking through this in a focal vm, I'm seeing some
strange things.
Starting with a clean disk, and just creating the backing device like
so:
make-bcache -B /dev/vdb
We see /dev/bcache0 get created with vdb as the backing device. Now,
after this, I see:
/dev/bcache/by-uuid/ -> ../../bcache0
This is unexpected. It should *only* appear once the bcache0 device is
actively cached; that is the bcache driver should not emit CACHED_UUID,
until the UUID is actually cached, which only happens once we've
registered a cache device with the bcache0 device. We need to look at
the kernel source to see if the SAUCE patch isn't quite right, or some
other part of bcache code has changed.
Next issue, blkid now detects the bcache dev.uuid, and this shows up
like so:
# udevadm test-builtin blkid /sys/class/block/vdb
...
vdb: Probe /dev/vdb with raid and offset=0
ID_FS_UUID=d7d7e025-b8d2-43cb-a5df-c240ba1418dc
ID_FS_UUID_ENC=d7d7e025-b8d2-43cb-a5df-c240ba1418dc
ID_FS_TYPE=bcache
ID_FS_USAGE=other
Note, it shows up as an FS_UUID, but it is *NOT* an fs_uuid; there is no
filesystem on this device at all at this time; it does have a bcache
superblock, note the FS_UUID matches the dev.uuid of the backing device.
# bcache-super-show /dev/vdb
sb.magicok
sb.first_sector 8 [match]
sb.csum 29D6774A332A280B [match]
sb.version 1 [backing device]
dev.label (empty)
dev.uuidd7d7e025-b8d2-43cb-a5df-c240ba1418dc
dev.sectors_per_block 1
dev.sectors_per_bucket 1024
dev.data.first_sector 16
dev.data.cache_mode 0 [writethrough]
dev.data.cache_state0 [detached]
cset.uuid 37b37bc1-e185-4825-8900-579df102b7d6
It's also curious that cset.uuid has a UUID, as there are no csets
created; IMO this is also a bug and it should be 0, null, or empty.
Now, here's where the udev fights over the links. The backing device,
vdb, triggers this rule in 60-persistent-storage.rules:
# by-label/by-uuid links (filesystem metadata)
ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*",
SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
This creates confusion due to the CACHED_UUID also being emitted from
the kernel, like so:
root@ubuntu:/run/udev/links# ls -al
/dev/disk/by-uuid/d7d7e025-b8d2-43cb-a5df-c240ba1418dc
lrwxrwxrwx 1 root root 9 May 21 16:39
/dev/disk/by-uuid/d7d7e025-b8d2-43cb-a5df-c240ba1418dc -> ../../vdb
root@ubuntu:/run/udev/links# ls -al
/dev/bcache/by-uuid/d7d7e025-b8d2-43cb-a5df-c240ba1418dc
lrwxrwxrwx 1 root root 13 May 21 16:39
/dev/bcache/by-uuid/d7d7e025-b8d2-43cb-a5df-c240ba1418dc -> ../../bcache0
There we have two devices both claiming the backing devices dev.uuid
value, with two different block names.
So, in summary, there are two problems to fix:
1) blkid on bcache devices should not report FS_UUID values when
FS_TYPE=bcache; there is no filesystem on it; this will need to be
discussed upstream
2) our kernel (need to check mainline vs. ours) emits CACHED_UUID when
the bcache0 is created, but no cache is attached: see this udev event:
KERNEL[2217.605118] change /devices/virtual/block/bcache0 (block)
ACTION=change
DEVPATH=/devices/virtual/block/bcache0
SUBSYSTEM=block
DRIVER=bcache
CACHED_UUID=d7d7e025-b8d2-43cb-a5df-c240ba1418dc
CACHED_LABEL=
DEVNAME=/dev/bcache0
DEVTYPE=disk
SEQNUM=5890
MAJOR=251
MINOR=0
** Changed in: linux (Ubuntu Groovy)
Status: Invalid => New
** Changed in: linux-signed (Ubuntu Groovy)
Status: Invalid => New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1861941
Title:
bcache by-uuid links disappear after mounting bcache0
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
@Balint I do not thing the fix you're released is correct, can you upload a new version without the scripts? Also, we should fix make-bcache -B to ensure that cset.uuid is not initialized; that may be why the kernel thinks it should emit the CACHED_UUID if the suerpblock of the device has a cset.uuid value (versus None). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878682] Re: IPv6 not enabled by default
In your dual stack example, can you run cloud-init collect-logs and attach the created tarball? In particular the cloud-init.log and what, if any network configuration you're supplying in your NoCloud datasource. > This tells me that someone enabled dhcp4 but not dhcp6 for cloud-init. We > need to make sure we have feature parity across the protocols. If you are not providing any network config to cloud-init, current the default fallback is to DHCP (ipv4) only; Ubuntu itself defaults to accepting IPv6 RAs, but asynchronously from boot. And for ipv6 config, I suspect you'll want to provide an netplan config with: network: version: 2 ethernets: primary: match: macaddress: 52:54:00:4d:27:e9 dhcp4: true dhcp6: true accept-ra: yes Which ensures that systemd-networkd-wait-online.service will wait the required time for querying and accepting any RA. ** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878682 Title: IPv6 not enabled by default To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1878682/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1879367] Re: Impossible to configure LVM when choosing manual partitioning mode
Thanks for submitting the bug. Curtin project is not currently part of the desktop installer. I've added the correct project (Ubiquity) for your request. ** Also affects: ubiquity (Ubuntu) Importance: Undecided Status: New ** Changed in: curtin (Ubuntu) Status: New => Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1879367 Title: Impossible to configure LVM when choosing manual partitioning mode To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1879367/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
That doesn't explain why they show up sometimes, but not all of the time. There are 3 devices in play here. * The backing device, let's say /dev/vda; this is where we want to store the data. * The caching device, let's say /dev/vdb; this holds the cache. * The bcache device; this only appears when a backing device is detected; via bcache-probe which reads the header on the disk and indicates it's a bcache device. Now, there are a few sequences that we see: 1) make-bcache -b /dev/vda -c /deb/vdb will create a bcache0 with the cache and backing dev at the same time and emit the CACHED_UUID change event which runs *after* 60-persistent-storage, and adds the bcache/by-uuid links 2) After a reboot (or if you tear down a bcache0 by stopping them in sysfs) we have several scenarios where the 2 real block devices are found: 2a) order: backing, bcacheN, caching In this scenario, vda will run through 60-persistent-storage.rules and blkid will discover ID_FS_TYPE=bcache, and generates only by-path, by-id links for the device, 69-bcache.rules will run bcache-register /dev/vda which informs the kernel that a backing device has been found. If the cache device has not yet been handled by udev (we don't know if a cache device exists, bcache can work without one) then the kernel will create a bcacheN device, but without the cache device being present, it will *NOT* emit a CHANGE event with CACHED_UUID value. Now when the cache device (vdb) is going through, 60-persistent-storage.rules, blkid will discover the same settings, TYPE=bcache, generate by-path, by-id symlinks. Then 69-bcache.rules runs, triggers a bcache-register on vdb registering the cache device in the kernel and because the superblock on the backing device specifies the cache device UUID it used the kernel can bind the cache device into the correct bcacheN device. A mainline kernel (no sauce patch) does not emit a CHANGE event with CACHED_UUID value. Our sauce patch changes the kernel to emit the CHANGE event with CACHED_UUID whenever we bind a cache device to an existing bcacheN precisely so the bcache/by-* links get generated. 2b) order: cache, backing, bcacheN Here, when vdb is plugged in, the cache device is registered in the kernel, but without a backing device, no bcacheN device is created yet. Then the backing is probed, as in 2a; the bcache-register command will register it with the kernel, and create a bcache0, and since the cache device is present, mainline kernels (and ours) will emit the CHANGE event with CACHED_UUID With this in-mind; it appears to me that in the focal case we're seeing the bcache driver CHANGE event with CACHED_UUID and then additional change events from the block layer around the bcache0 device (but not emitted from the bcache driver code) So I'd like to understand why ... I expect those events to occur but not before the driver change event. And maybe that's the race here The happy path: 1. udev processing CHANGE on backing device, it's bcache so we register it 2. kernel bcache driver creates a bcache0 block device (udev ADD event) 3. udev processing CHANGE on bcache0 device 4. udev processing CHANGE on cache device, it's bcache so we register it 5. kernel bcache driver emits CHANGE event with CACHED_UUID 6. udev processing CHANGE on bcache0 device with CACHED_UUID The bad path 1. udev processing CHANGE on cache device, it's bcache so we register it 2. udev processing CHANGE on backing device, it's bcache so we register it 3. kernel bcache driver creates a bcache0 block device (udev ADD event) 4. kernel bcache driver emits CHANGE event with CACHED_UUID 5. udev processing CHANGE on bcache0 device with CACHED_UUID 6. udev processing CHANGE on bcache0 device without CACHED_UUID if the bcache change event occurs before the block layer change events, then the database does get clobbered ... but we've an incomplete trace; we should see pairs of events, one from KERNEL, and then one from UDEV. We should definitely get a complete capture for this bug. w.r.t the fix; I've a few concerns: 1) bcache-keep-symlinks relies on /dev/bcache/by-* links to exist, if they are removed, then the keep won't work. 2) The removal of the /dev/bcache/by-uuid/ happens during processing of 60-persistent-storage.rules, which will remove the link before bcache-keep-symlinks can run (it's in 69-bcache.rules) 3) since we cannot rely on the existing links, instead we should read and parse the bcache superblock devices, much like the one I put together (and you reminded me I wrote) https://github.com/g2p/bcache-tools/pull/29/files -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1861941 Title: bcache by-uuid links disappear after mounting bcache0 To manage
[Bug 1861941] Re: bcache by-uuid links disappear after mounting bcache0
I guess I don't understand why we see this in focal. The two events in
Colin's trace always happen on any Ubuntu kernel. We should see if we
can get another udev trace on bionic that captures both CHANGE events,
one will be from the bcache driver itself, and one is from the block
layer. THe order and content of the change event matter.
Another thing I don't understand is why does udev drop the a symlink created by
another rule? This seems like the core issue.
Looking at systemd/udev source code; udev will do a
FOREACH_DEVICE_DEVLINK and check if the name is in the database file for
the device. Its not clear to me yet how or when the database file get's
written.
The other question I have is: if we reversed the order of the focal
CHANGE events, wouldn't we see just the opposite happen (the
driver=bcache change event would not have all of the devlinks from a
blkid probe) and all of the /dev/disk/by-{id, uuid, ...} get removed
when running
I think the patch you're proposing should work; but I don't think we've
root caused why the link gets removed in the first place. Once we
understand the root cause, I think we can better understand what a fix
should look like.
Lastly, I think we might also test this out on mainline kernel; I wonder if our
SAUCE patch for emitting the CACHED_UUID needs an update (or could be dropped).
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1861941
Title:
bcache by-uuid links disappear after mounting bcache0
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/bcache-tools/+bug/1861941/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1875560] Re: subiquity crashes on configuring existing partition
I believe the current failure you see is this one: https://bugs.launchpad.net/subiquity/+bug/1878890 A fix has been created, it's getting reviewed and should land soon. Once merged subiquity can pull in the fix. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1875560 Title: subiquity crashes on configuring existing partition To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/subiquity/+bug/1875560/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1639955] Re: bad test for snappy systems
This issue was fixed a while back in: https://github.com/canonical/cloud-init/commit/4bcc94730 cloud-init retains the legacy bits, but prefers the use of os-release file or kernel commandline to for checking for UbuntuCore. ** Changed in: cloud-init (Ubuntu) Status: Confirmed => Fix Released ** Changed in: cloud-init Status: Confirmed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1639955 Title: bad test for snappy systems To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1639955/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878752] Re: vgcreate fails on /dev/disk/by-dname block devices
I would very much agree with xnox (it's a duplicate) (and Dan, nothing for curtin to do); curtin generated dname rules rely upon pointing to a /dev/bcache/by- uuid/* symlink which is currently broken per https://bugs.launchpad.net/ubuntu/+source/linux-signed/+bug/1861941 which at this time points some issue in udev itself (the kernel emits all of the correct uevents we expect). And as James' workaround shows; it's *not* always happening; a rescan can "restore" the links; but that's not 100% reliable. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1878752 Title: vgcreate fails on /dev/disk/by-dname block devices To manage notifications about this bug go to: https://bugs.launchpad.net/charm-ceph-osd/+bug/1878752/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1878890] Re: [Ubuntu Server 20.04 LTS]: Failed Install (subiquity...install_fail/add_info) during partitioning
>From the crash:
Verifying /dev/sda2 exists
Running command ['sfdisk', '--json', '/dev/sda'] with allowed return codes [0]
(capture=True)
Verifying /dev/sda2 size, expecting 86564143104 bytes, found 86564143104 bytes
Device /dev/sda2 ptable entry: {
"node": "/dev/sda2",
"size": 169070592,
"start": 61442048,
"type": "83"
}
Verifying /dev/sda2 partition flag, expecting linux, found None
An error occured handling 'partition-sda2': RuntimeError - Verifying /dev/sda2
partition flag, expecting linux, found None
Curtin has a logic bug here for msdos ptable and primary partitions; we
should lookup the type found in ptable_uuid_to_flag_entry().
** Also affects: curtin (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1878890
Title:
[Ubuntu Server 20.04 LTS]: Failed Install
(subiquity...install_fail/add_info) during partitioning
To manage notifications about this bug go to:
https://bugs.launchpad.net/subiquity/+bug/1878890/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876848] Re: Installation of Focal on a linux raid consistently fails
@Stian No worries; it's nothing you configured/changed as far as I can tell. We look up device paths to disk via serial or wwn; as it turns out your two disks have different serials, but a duplicate (and invalid WWN); so we'll need to fix our code to ignore these invalid WWN and not include them in the storage config; I suspect if we removed two 'wwn' keys from the storage config, all would work fine. Thanks for the crash, we have enough data here to fix the issue. @Michael, in block-discover, we should not include the wwn key if the value is invalid (0x... ) If we omit the key then we'll have 'serial' which does differ and we will find the correct block devices. ** Also affects: curtin (Ubuntu) Importance: Undecided Status: New ** Changed in: curtin (Ubuntu) Importance: Undecided => High ** Changed in: curtin (Ubuntu) Status: New => Triaged -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876848 Title: Installation of Focal on a linux raid consistently fails To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/curtin/+bug/1876848/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1876139] Re: Groovy cloud-images failing during growpart
** Attachment added: "Source package tarball for upload" https://bugs.launchpad.net/cloud-utils/+bug/1876139/+attachment/5367036/+files/cloud-utils-groovy-source-pkg.tar -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1876139 Title: Groovy cloud-images failing during growpart To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-images/+bug/1876139/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
