Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
I solved the issue for my environment. I observed the armhf container
did not receive dnsmasq's UDP replies. Examining /usr/libexec/lxc/lxc-
net I identified this line
iptables $use_iptables_lock -t mangle -A POSTROUTING -o ${LXC_BRIDGE}
-p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
as something special and added its equivalent to my firewall
configuration. Since then networking works perfectly well also for the
armhf container.
I believe the special purpose of this iptables rule should be pointed
out not sure where.
Antonio, our questions helped me to look in the right direction. Thanks
for your patience.
signature.asc
Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
I tracked the issue down to the following point: Networking works only for native containers with my usual local setup made up of a bridge configured in /etc/network/interfaces, dnsmasq listing on that bridge and arno-iptables-firewall doing the masquerading and forwarding. Networking works for native and armhf containers with using lxc-net and my firewall disabled. Networking fails for native and armhf containers with using lxc-net and my firewall enabled. So my questions are: What's the difference between using lcx-net alone and my home-grown setup (which works perfectly well also for qemu and virtualbox VMs) with regards to non-native containers? Does lxc-net anything specific regarding a containers architecture? Should this be a bug against lxc then? Off-topic: At present my email provider does not handle sender validiation properly, so the mail server for debian.org refuses my mails. Luckily they are accepted at bugs.debian.org. Sven signature.asc Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
> Does networking work in n ative plain lxc container? Networking works with plain native containers as is does with debci generated native ones. root@e580sg:~# lxc-start -n amd64 root@e580sg:~# lxc-attach -n amd64 root@amd64:~# LANG=C apt update Get:1 http://security.debian.org stable/updates InRelease [65.4 kB] Hit:2 http://deb.debian.org/debian stable InRelease Get:3 http://deb.debian.org/debian stable/main Translation-en [5968 kB] Get:4 http://security.debian.org stable/updates/main amd64 Packages [233 kB] Get:5 http://security.debian.org stable/updates/main Translation-en [125 kB] Fetched 6392 kB in 5s (1329 kB/s) Reading package lists... Done Building dependency tree... Done All packages are up to date. root@amd64:~# exit exit root@e580sg:~# lxc-stop -n amd64 Note: lxc-stop terminates quickly and without any error message. > What does your/etc/lxc/default.conf look like? lxc.net.0.type = veth lxc.net.0.link = br0 lxc.net.0.flags = up lxc.apparmor.profile = generated lxc.apparmor.allow_nesting = 1 Sven signature.asc Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
On Sat, Sep 26, 2020 at 04:09:34PM +0200, Sven Geuer wrote: > Further tests show that networking does not seem to work: > > root@e580sg:~# lxc-start -n armhf > root@e580sg:~# lxc-attach -n armhf > root@armhf:~# LANG=C apt update > Err:1 http://security.debian.org stable/updates InRelease > Temporary failure resolving 'security.debian.org' > Err:2 http://deb.debian.org/debian stable InRelease > Temporary failure resolving 'deb.debian.org' > Reading package lists... Done > Building dependency tree... Done > All packages are up to date. > W: Failed to fetch http://deb.debian.org/debian/dists/stable/InRelease > Temporary failure resolving 'deb.debian.org' > W: Failed to fetch > http://security.debian.org/dists/stable/updates/InRelease Temporary > failure resolving 'security.debian.org' > W: Some index files failed to download. They have been ignored, or old > ones used instead. > root@armhf:~# exit > exit > root@e580sg:~# lxc-stop -n armhf > lxc-stop: armhf: commands_utils.c: lxc_cmd_sock_rcv_state: 72 Resource > temporarily unavailable - Failed to receive message Does networking work in n ative plain lxc container? What does your /etc/lxc/default.conf look like? signature.asc Description: PGP signature
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Further tests show that networking does not seem to work: root@e580sg:~# lxc-start -n armhf root@e580sg:~# lxc-attach -n armhf root@armhf:~# LANG=C apt update Err:1 http://security.debian.org stable/updates InRelease Temporary failure resolving 'security.debian.org' Err:2 http://deb.debian.org/debian stable InRelease Temporary failure resolving 'deb.debian.org' Reading package lists... Done Building dependency tree... Done All packages are up to date. W: Failed to fetch http://deb.debian.org/debian/dists/stable/InRelease Temporary failure resolving 'deb.debian.org' W: Failed to fetch http://security.debian.org/dists/stable/updates/InRelease Temporary failure resolving 'security.debian.org' W: Some index files failed to download. They have been ignored, or old ones used instead. root@armhf:~# exit exit root@e580sg:~# lxc-stop -n armhf lxc-stop: armhf: commands_utils.c: lxc_cmd_sock_rcv_state: 72 Resource temporarily unavailable - Failed to receive message signature.asc Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Hello Antonio, > e.g. does this work: > > sudo lxc-create --template=debian --name=armhf -- --arch=armhf > sudo lxc-start --name=armhf > sudo lxc-attach --name=armhf > > ? It seems to work in some way. But there are some errors at the end of the installation process. Also, lxc-stop takes several seconds to complete and it comes back with an error. Here's the relevant output: root@e580sg:~# lxc-create --template=debian --name=armhf -- -- arch=armhf debootstrap ist /usr/sbin/debootstrap Checking cache download in /var/cache/lxc/debian/rootfs-stable-armhf ... Downloading debian minimal ... I: Target architecture can be executed I: Retrieving InRelease I: Checking Release signature I: Valid Release signature (key id 6D33866EDD8FFA41C0143AEDDCC9EFBF77E11517) I: Retrieving Packages I: Validating Packages I: Resolving dependencies of required packages... I: Resolving dependencies of base packages... I: Checking component main on http://deb.debian.org/debian... [...] I: Base system installed successfully. Download complete. Copying rootfs to /var/lib/lxc/armhf/rootfs...Generating locales (this might take a while)... de_DE.UTF-8... done de_DE.UTF-8... done Generation complete. update-rc.d: error: cannot find a LSB script for checkroot.sh update-rc.d: error: cannot find a LSB script for umountfs Failed to disable unit, unit hwclock.sh.service does not exist. update-rc.d: error: cannot find a LSB script for hwclockfirst.sh Creating SSH2 RSA key; this may take some time ... 2048 SHA256:eyTvbzzRV0YzdRVk9sFLWp9dzK0nRuUPgN2k6QMpkMY root@e580sg (RSA) Creating SSH2 ECDSA key; this may take some time ... 256 SHA256:CfbEORbaL5SptfEnhIQDdg3M/3tNkRPX1mkL5FwUyOE root@e580sg (ECDSA) Creating SSH2 ED25519 key; this may take some time ... 256 SHA256:rGf9JbAsSf8Go+5CZGLQH5Vag4reEEey/PjEWfVl8Vg root@e580sg (ED25519) invoke-rc.d: could not determine current runlevel invoke-rc.d: policy-rc.d denied execution of start. Current default time zone: 'Etc/UTC' Local time is now: Sat Sep 26 13:31:23 UTC 2020. Universal Time is now: Sat Sep 26 13:31:23 UTC 2020. root@e580sg:~# lxc-start --name=armhf root@e580sg:~# lxc-attach --name=armhf root@armhf:~# uname -a Linux armhf 5.8.0-2-amd64 #1 SMP Debian 5.8.10-1 (2020-09-19) armv7l GNU/Linux root@armhf:~# exit exit root@e580sg:~# lxc-stop --name=armhf lxc-stop: armhf: commands_utils.c: lxc_cmd_sock_rcv_state: 72 Resource temporarily unavailable - Failed to receive message Hope this helps. Sven signature.asc Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
On Fri, Sep 25, 2020 at 10:08:40PM +0200, Sven Geuer wrote: > Hello Antonio, > > I have no issue with containers of the native amd64 architecture of my > system. Setting up and using a i386 container also works flawlessly. I > have been using them even before trying to setup a container of a > foreign architecture. > > I seem to have a general issue to setup containers of a foreign > architecture, armhf is a mere example. I repeated a armhf setup. It > keeps failing, while the error looks different now compared to my first > post. It presents itself as a network access error: > > [...] > Running setup script /usr/share/autopkgtest/setup-commands/setup- > testbed... > /usr/bin/sh: Attempting to set up Debian/Ubuntu apt sources > automatically > /usr/bin/sh: Distribution assumed to resemble Debian > Err:1 http://deb.debian.org/debian unstable InRelease > Temporary failure resolving 'deb.debian.org' > Reading package lists... > W: Failed to fetch > http://deb.debian.org/debian/dists/unstable/InRelease Temporary > failure resolving 'deb.debian.org' > [...] > > Please see attach a complete log of the setup run. Hope this helps to > track down what going on. I already knew you couldn't setup a debci container, that's why I asked if you are able to create and start a plain lxc container in a foreign architecture, so we can track down whether the issue is in debci, lxc, or something else that is broken on your end (since it works for me). e.g. does this work: sudo lxc-create --template=debian --name=armhf -- --arch=armhf sudo lxc-start --name=armhf sudo lxc-attach --name=armhf ? signature.asc Description: PGP signature
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Hello Antonio, I have no issue with containers of the native amd64 architecture of my system. Setting up and using a i386 container also works flawlessly. I have been using them even before trying to setup a container of a foreign architecture. I seem to have a general issue to setup containers of a foreign architecture, armhf is a mere example. I repeated a armhf setup. It keeps failing, while the error looks different now compared to my first post. It presents itself as a network access error: [...] Running setup script /usr/share/autopkgtest/setup-commands/setup- testbed... /usr/bin/sh: Attempting to set up Debian/Ubuntu apt sources automatically /usr/bin/sh: Distribution assumed to resemble Debian Err:1 http://deb.debian.org/debian unstable InRelease Temporary failure resolving 'deb.debian.org' Reading package lists... W: Failed to fetch http://deb.debian.org/debian/dists/unstable/InRelease Temporary failure resolving 'deb.debian.org' [...] Please see attach a complete log of the setup run. Hope this helps to track down what going on. Thanks, Sven Am Montag, den 21.09.2020, 11:47 -0300 schrieb Antonio Terceiro: > Control: tag -1 + moreinfo > > On Sun, Sep 20, 2020 at 08:32:50PM +0200, Sven Geuer wrote: > > Dear Maintainer, > > > > I am missing someone has taken a look into this issue. I'd > > appreciate > > to get some feedback. > > > > Please let me know what further input from my side may be helpful. > > I just tried it here, and it just worked for me. Are you able to > start > regular lxc containers? Starting testbed setup for unstable: Fri Sep 25 21:55:48 CEST 2020 lxc-create: autopkgtest-unstable-armhf.new: storage/btrfs.c: btrfs_create: 961 Inappropriate ioctl for device - Failed to create btrfs subvolume "/var/lib/lxc/autopkgtest-unstable-armhf.new/rootfs" lxc-create: autopkgtest-unstable-armhf.new: storage/zfs.c: zfs_create: 761 Failed to create zfs dataset "zfs:lxc/autopkgtest-unstable-armhf.new": lxc-create: autopkgtest-unstable-armhf.new: utils.c: run_command_internal: 1669 Failed to exec command lxc-create: autopkgtest-unstable-armhf.new: storage/lvm.c: do_lvm_create: 204 Failed to create logical volume "autopkgtest-unstable-armhf.new": Volume group "lxc" not found. Cannot process volume group lxc lxc-create: autopkgtest-unstable-armhf.new: storage/lvm.c: lvm_create: 676 Error creating new logical volume "lvm:/dev/lxc/autopkgtest-unstable-armhf.new" of size "1073741824 bytes" debootstrap is /usr/sbin/debootstrap Checking cache download in /var/cache/lxc/debian/rootfs-unstable-armhf ... Downloading debian minimal ... I: Target architecture can be executed I: Retrieving InRelease I: Checking Release signature I: Valid Release signature (key id 0146DC6D4A0B2914BDED34DB648ACFD622F3D138) I: Retrieving Packages I: Validating Packages I: Resolving dependencies of required packages... I: Resolving dependencies of base packages... I: Checking component main on http://deb.debian.org/debian... I: Retrieving libacl1 2.2.53-8 I: Validating libacl1 2.2.53-8 I: Retrieving adduser 3.118 I: Validating adduser 3.118 I: Retrieving libapparmor1 2.13.4-3 I: Validating libapparmor1 2.13.4-3 I: Retrieving apt 2.1.10 I: Validating apt 2.1.10 I: Retrieving libapt-pkg6.0 2.1.10 I: Validating libapt-pkg6.0 2.1.10 I: Retrieving libargon2-1 0~20171227-0.2 I: Validating libargon2-1 0~20171227-0.2 I: Retrieving libattr1 1:2.4.48-5 I: Validating libattr1 1:2.4.48-5 I: Retrieving libaudit-common 1:2.8.5-3 I: Validating libaudit-common 1:2.8.5-3 I: Retrieving libaudit1 1:2.8.5-3+b1 I: Validating libaudit1 1:2.8.5-3+b1 I: Retrieving base-files 11 I: Validating base-files 11 I: Retrieving base-passwd 3.5.47 I: Validating base-passwd 3.5.47 I: Retrieving bash 5.0-7 I: Validating bash 5.0-7 I: Retrieving libdns-export1110 1:9.11.19+dfsg-1 I: Validating libdns-export1110 1:9.11.19+dfsg-1 I: Retrieving libisc-export1105 1:9.11.19+dfsg-1 I: Validating libisc-export1105 1:9.11.19+dfsg-1 I: Retrieving libbz2-1.0 1.0.8-4 I: Validating libbz2-1.0 1.0.8-4 I: Retrieving libdebconfclient0 0.254 I: Validating libdebconfclient0 0.254 I: Retrieving coreutils 8.32-4 I: Validating coreutils 8.32-4 I: Retrieving libcryptsetup12 2:2.3.4-1 I: Validating libcryptsetup12 2:2.3.4-1 I: Retrieving dash 0.5.10.2-7 I: Validating dash 0.5.10.2-7 I: Retrieving libdb5.3 5.3.28+dfsg1-0.6 I: Validating libdb5.3 5.3.28+dfsg1-0.6 I: Retrieving debconf 1.5.74 I: Validating debconf 1.5.74 I: Retrieving debian-archive-keyring 2019.1 I: Validating debian-archive-keyring 2019.1 I: Retrieving debianutils 4.11.1 I: Validating debianutils 4.11.1 I: Retrieving runit-helper 2.9.0 I: Validating runit-helper 2.9.0 I: Retrieving dialog 1.3-20190808-1 I: Validating dialog 1.3-20190808-1 I: Retrieving diffutils 1:3.7-3 I: Validating diffutils 1:3.7-3 I: Retrieving dpkg 1.20.5 I: Validating dpkg 1.20.5 I: Retrieving e2fsprogs 1.45.6-1 I: Validating e2fsprogs 1.45.6-1 I: Retrieving libcom-err2 1.45.6-1 I: Validating libcom-err2 1.45.6-1 I: Retrieving lib
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Control: tag -1 + moreinfo On Sun, Sep 20, 2020 at 08:32:50PM +0200, Sven Geuer wrote: > Dear Maintainer, > > I am missing someone has taken a look into this issue. I'd appreciate > to get some feedback. > > Please let me know what further input from my side may be helpful. I just tried it here, and it just worked for me. Are you able to start regular lxc containers? signature.asc Description: PGP signature
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Dear Maintainer, I am missing someone has taken a look into this issue. I'd appreciate to get some feedback. Please let me know what further input from my side may be helpful. Regards, Sven signature.asc Description: This is a digitally signed message part
Bug#958910: debci: 'debci setup -a armhf' fails to set up an lxc container on an amd64 host
Package: debci Version: 2.11 Severity: normal Dear Maintainer, I intended to set up an armhf lxc container by running debci setup -a armhf as root on my amd64 system. The installation process terminated with [...] Timed out waiting for container to boot lxc-stop: autopkgtest-unstable-armhf.new: tools/lxc_stop.c: main: 191 autopkgtest-unstable-armhf.new is not running lxc-destroy: autopkgtest-unstable-armhf.new: tools/lxc_destroy.c: main: 271 Destroyed container autopkgtest-unstable-armhf.new In /var/log/syslog I encountered these lines probably of relevance Apr 26 17:16:25 e580sg kernel: [14086.041927] audit: type=1400 audit(1587914185.032:51): apparmor="STATUS" operation="profile_load" profile="/usr/bin/lxc-start" name="lxc-autopkgtest-unstable-armhf.new_" pid=88028 comm="apparmor_parser" Apr 26 17:16:25 e580sg NetworkManager[686]: [1587914185.0606] manager: (vethSB9884): new Veth device (/org/freedesktop/NetworkManager/Devices/22) Apr 26 17:16:25 e580sg kernel: [14086.064571] br0: port 2(vethS5LJSK) entered blocking state Apr 26 17:16:25 e580sg kernel: [14086.064573] br0: port 2(vethS5LJSK) entered disabled state Apr 26 17:16:25 e580sg kernel: [14086.064617] device vethS5LJSK entered promiscuous mode Apr 26 17:16:25 e580sg kernel: [14086.064692] br0: port 2(vethS5LJSK) entered blocking state Apr 26 17:16:25 e580sg kernel: [14086.064693] br0: port 2(vethS5LJSK) entered forwarding state Apr 26 17:16:25 e580sg kernel: [14086.066265] br0: port 2(vethS5LJSK) entered disabled state Apr 26 17:16:25 e580sg systemd-udevd[88029]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable. Apr 26 17:16:25 e580sg NetworkManager[686]: [1587914185.0618] manager: (vethS5LJSK): new Veth device (/org/freedesktop/NetworkManager/Devices/23) Apr 26 17:16:25 e580sg systemd-udevd[88029]: Using default interface naming scheme 'v245'. Apr 26 17:16:25 e580sg systemd-udevd[88029]: Could not set Alias=, MACAddress= or MTU= on vethSB9884: No such device Apr 26 17:16:25 e580sg systemd-udevd[88029]: vethSB9884: Could not apply link config, ignoring: No such device Apr 26 17:16:25 e580sg systemd-udevd[88030]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable. Apr 26 17:16:25 e580sg systemd-udevd[88030]: Using default interface naming scheme 'v245'. Apr 26 17:16:25 e580sg kernel: [14086.092692] eth0: renamed from vethSB9884 Apr 26 17:16:25 e580sg gnome-shell[2393]: Removing a network device that was not added Apr 26 17:16:25 e580sg NetworkManager[686]: [1587914185.1205] device (vethS5LJSK): carrier: link connected Apr 26 17:16:25 e580sg NetworkManager[686]: [1587914185.1209] device (br0): carrier: link connected Apr 26 17:16:25 e580sg kernel: [14086.124443] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready Apr 26 17:16:25 e580sg kernel: [14086.124480] IPv6: ADDRCONF(NETDEV_CHANGE): vethS5LJSK: link becomes ready Apr 26 17:16:25 e580sg kernel: [14086.124553] br0: port 2(vethS5LJSK) entered blocking state Apr 26 17:16:25 e580sg kernel: [14086.124555] br0: port 2(vethS5LJSK) entered forwarding state Apr 26 17:16:25 e580sg kernel: [14086.152175] audit: type=1400 audit(1587914185.144:52): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="/usr/bin/lxc-start" name="/proc/sys/kerne l/random/boot_id" pid=88031 comm="lxc-start" srcname="/dev/.lxc-boot-id" flags="rw, bind" Apr 26 17:16:25 e580sg kernel: [14086.154374] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist. Apr 26 17:16:25 e580sg kernel: [14086.248964] br0: port 2(vethS5LJSK) entered disabled state Apr 26 17:16:25 e580sg kernel: [14086.250781] device vethS5LJSK left promiscuous mode Apr 26 17:16:25 e580sg kernel: [14086.250785] br0: port 2(vethS5LJSK) entered disabled state Apr 26 17:16:25 e580sg NetworkManager[686]: [1587914185.2731] device (vethS5LJSK): released from master device br0 Apr 26 17:16:25 e580sg gnome-shell[2393]: Removing a network device that was not added Apr 26 17:16:25 e580sg kernel: [14086.394930] audit: type=1400 audit(1587914185.384:53): apparmor="STATUS" operation="profile_remove" profile="/usr/bin/lxc-start" name="lxc-autopkgtest-unstable- armhf.new_" pid=88076 comm="apparmor_parser" These are the installed qemu packages $ LANG=C dpkg -l 'qemu-*' | grep -v '^un' Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name VersionArchitecture Description +++--==--== ii qemu-efi-aarch64 0.0~20200229-2 all UEFI firmware for 64-bit ARM virtual machines ii qemu-efi-arm 0.0~20200229-2 all UEFI firmware for 32-bit ARM virtual machines ii qemu-kvm 1:4.2-6amd64QEMU Full virtualization
