Bug#960729: More issues trying to create an Ubuntu focal image
On 2024-04-11 15:25, Paride Legovini wrote: > On 2024-04-11 08:35, Christian Kastner wrote: > Ubuntu did indeed switch to something else: that's netplan.io. > On a Bionic system: > > $ apt show netplan.io > Package: netplan.io > Version: 0.99-0ubuntu3~18.04.5 > Priority: important Oh, that explains ifupdown moving to universe, I guess... > So we have another option: teach setup-testbed how to configure > netplan. > This would be a more realistic setup for a modern Ubuntu system, > and won't need any extra dependency outside of what debootstrap > installs automatically. I think "more realistic" is a strong argument for a test environment. And if the package is already installed, adding the configuring step seems the simplest solution. Best, Christian
Bug#960729: More issues trying to create an Ubuntu focal image
On 2024-04-11 08:35, Christian Kastner wrote: > Control: tags -1 - pending > > On 2024-04-08 15:21, Paride Legovini wrote: >> Fixed in master by: >> >> https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/315 > > Sadly, it turns out that this wasn't the fix, at least not in a wider sense. > > Yes, images can be built now, but without ifupdown their network > interface is left unconfigured, and thus autopkgtests can't download > packages. > > With the move of ifupdown to universe, I was assuming that Ubuntu did > things differently. The cloud images *do* things differently, namely > they have systemd-networkd. But autopkgtest allows for alternative init > systems, so we can't rely on that. Ubuntu did indeed switch to something else: that's netplan.io. On a Bionic system: $ apt show netplan.io Package: netplan.io Version: 0.99-0ubuntu3~18.04.5 Priority: important and by running e.g. sudo tools/autopkgtest-build-qemu --mirror=http://archive.ubuntu.com/ubuntu/ jammy jammy-qemu.img I see it being installed, but left unconfigured. Now, when we have ifupdown, what is configuring it? It's setup-testbed: printf 'auto %s\niface %s inet dhcp\n' "$IFACE" "$IFACE" >> "$root/etc/network/interfaces.d/$IFACE" So we have another option: teach setup-testbed how to configure netplan. This basically consists in dropping a yaml under /etc/netplan containing something like: network: version: 2 renderer: networkd ethernets: eth0: dhcp4: yes This would be a more realistic setup for a modern Ubuntu system, and won't need any extra dependency outside of what debootstrap installs automatically. Or we could enable universe during debootstrap, install ifupdown and hope that it will keep playing fine with netplan, as you suggested. Maybe we should just take this simpler route, I am still unsure, input is welcome. Paride
Bug#960729: More issues trying to create an Ubuntu focal image
On 2024-04-11 09:27, Paul Gevers wrote: >> (2) Modify setup-commands/setup-testbed and generally move ifupdown >> installation there. This would require re-ordering stuff, as >> interface configuration is currently performed *before* APT >> configuration, when in needs to be after (otherwise ifupdown can't >> be downloaded from universe). > > I'm against this route. I think we should require images to have > networking to not grow the setup script even more than needed. > Networking is nothing special, it should just be there. I concur. From my first experiments with this, the script just felt like the wrong place. It involves guessing the testbed type, and it would require somewhat nuanced restructuring of the script. And as long as universe is enabled unconditionally in the post-build image customization by setup-command, limiting the build to main doesn't seem to gain much. Best, Christian
Bug#960729: More issues trying to create an Ubuntu focal image
Hi On 11-04-2024 8:35 a.m., Christian Kastner wrote: Yes, images can be built now, but without ifupdown their network interface is left unconfigured, and thus autopkgtests can't download packages. Somewhere in the back of my mind I was fearing this. So ifupdown seems to be needed, and this poses an interesting problem with at least the following possible solutions: (1) Enable universe in autopkgtest-build-qemu after all, as in [1] Ack. (2) Modify setup-commands/setup-testbed and generally move ifupdown installation there. This would require re-ordering stuff, as interface configuration is currently performed *before* APT configuration, when in needs to be after (otherwise ifupdown can't be downloaded from universe). I'm against this route. I think we should require images to have networking to not grow the setup script even more than needed. Networking is nothing special, it should just be there. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#960729: More issues trying to create an Ubuntu focal image
Control: tags -1 - pending On 2024-04-08 15:21, Paride Legovini wrote: > Fixed in master by: > > https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/315 Sadly, it turns out that this wasn't the fix, at least not in a wider sense. Yes, images can be built now, but without ifupdown their network interface is left unconfigured, and thus autopkgtests can't download packages. With the move of ifupdown to universe, I was assuming that Ubuntu did things differently. The cloud images *do* things differently, namely they have systemd-networkd. But autopkgtest allows for alternative init systems, so we can't rely on that. So ifupdown seems to be needed, and this poses an interesting problem with at least the following possible solutions: (1) Enable universe in autopkgtest-build-qemu after all, as in [1] (2) Modify setup-commands/setup-testbed and generally move ifupdown installation there. This would require re-ordering stuff, as interface configuration is currently performed *before* APT configuration, when in needs to be after (otherwise ifupdown can't be downloaded from universe). Note that (1) and (2) don't exclude each other. One could do (1) to fix things short-term, then implement (2). And of course, there could be other solutions. And though I'm doing more and more Ubuntu stuff, I'm not yet familiar enough to propose better ideas. Best, Christian PS: Apologies for not discovering this right away, I was having troubles getting images to boot, but that's fixed now. [1] https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/309
Bug#960729: More issues trying to create an Ubuntu focal image
Control: tags -1 pending Fixed in master by: https://salsa.debian.org/ci-team/autopkgtest/-/merge_requests/315
Bug#960729: More issues trying to create an Ubuntu focal image
Control: block -1 by 951766 On Sat, 09 Oct 2021 at 22:27:57 +0200, Daniel Leidert wrote: > 2021-10-09 22:02:41 DEBUG STDERR: grub-install: unrecognized option > '--force-extra-removable' This part is a vmdb2 bug and is tracked separately, as #951766. If debci continues to use vmdb2 (directly or via autopkgtest-build-qemu, I don't know which one it does), then debci will be unable to support creating Ubuntu images until vmdb2 does. autopkgtest-buildvm-ubuntu-cloud, which downloads an official Ubuntu cloud image and modifies it in-place, might be a better way to get autopkgtest images to run Ubuntu on qemu (it also doesn't need root if run by someone with access to /dev/kvm, because it does all its privileged stuff by booting the image in a temporary qemu instance, which might be helpful). Doing the equivalent with Debian's official cloud images as a base is on my list, but it is not a short list, so it's anyone's guess when/whether I'll get there. smcv
Bug#960729: More issues trying to create an Ubuntu focal image
Hi there, I manually edited /usr/bin/autopkgtest-build-qemu and added: debootstrap['mirror'] = mirror debootstrap['target'] = 'root' +debootstrap['components'] = ['main', 'universe'] and got a little bit further. But then vmdb2 gets in the way: 2021-10-09 22:02:41 INFO Exec: ['chroot', '/tmp/tmpsued_j83', 'grub-install', '--target=i386-pc', '--no-nvram', '--force-extra-removable', '--no-floppy', '--modules=part_msdos part_gpt', '--grub-mkdevicemap=/boot/grub/device.map', '/dev/loop0'] 2021-10-09 22:02:41 DEBUG STDOUT: 2021-10-09 22:02:41 DEBUG STDERR: grub-install: unrecognized option '--force-extra-removable' This is due to /usr/lib/python3/dist-packages/vmdb/plugins/grub_plugin.py: vmdb.runcmd_chroot( chroot, [ "grub-install", "--target=" + grub_target, "--no-nvram", "--force-extra-removable", "--no-floppy", "--modules=part_msdos part_gpt", "--grub-mkdevicemap=/boot/grub/device.map", image_dev, ], ) After manually fixing this, it again fails when it comes to this: 2021-10-09 22:15:13 INFO Exec: ['sh', '-ec', 'export AUTOPKGTEST_BUILD_QEMU=1; /usr/share/debci/backends/qemu/customize.sh "$ROOT"'] 2021-10-09 22:15:14 DEBUG STDOUT: 2021-10-09 22:15:14 DEBUG STDERR: + readlink -f /usr/share/debci/backends/qemu/customize.sh + dirname /usr/share/debci/backends/qemu/customize.sh [..] + grep -q ubuntu.com /usr/share/debootstrap/scripts/focal + distro=ubuntu + [ ubuntu = debian ] + proxy_opt= + chroot /tmp/tmpm4s6f9l_ apt-config shell PROXY Acquire::http::Proxy + RES= + eval /usr/share/debci/backends/qemu/customize.sh: 43: PROXY: parameter not set 2021-10-09 22:15:14 ERROR Program failed: 2 Then I dropped the proxy stuff from /usr/share/debci/backends/qemu/customize.sh and ran it again and it finally worked. Regards, Daniel -- Regards, Daniel Leidert | https://www.wgdd.de/ GPG-Key RSA4096 / BEED4DED5544A4C03E283DC74BCD0567C296D05D GPG-Key ED25519 / BD3C132D8B3805D1808123AB7ACE00941E338C78 https://www.fiverr.com/dleidert https://www.patreon.com/join/dleidert signature.asc Description: This is a digitally signed message part
