Your message dated Thu, 12 May 2022 09:26:46 +0200
with message-id <[email protected]>
and subject line Re: netplan.io autopkgtest tmpfails on ci.d.n in testing
has caused the Debian Bug report #1000137,
regarding netplan.io autopkgtest tmpfails on ci.d.n in testing
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1000137: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1000137
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Source: netplan.io
Version: 0.103-3
Severity: important
Dear maintainer,
Somewhere between 2021-11-06 and 2021-11-12 the autopkgtest of
netplan.io has regressed on the ci.debian.net infrastructure on
ppc64el. It started to tmpfail after about 15 to 20 minutes. As
tmpfails are immediately retried by the Debian migration software,
this is causing quite some load. Do you know what goes wrong? Can we
help identify how this is possible?
I'm putting netplan.io on the reject_list for ppc64el for now. Please
reach out if you need our help to investigate.
I have diffed the artifacts for you in the attachment between the last
test that passed and the first that tmpfailed.
The end of the log looks like:
autopkgtest [20:28:47]: test cloud-init: ./debian/tests/prepare-testbed.sh &&
./debian/tests/cloud-init.sh
autopkgtest [20:28:47]: test cloud-init: [-----------------------
+ rm -f /etc/network/interfaces
+ systemctl unmask systemd-networkd.service
+ systemctl unmask systemd-networkd.socket
+ systemctl unmask systemd-networkd-wait-online.service
+ systemctl enable --now systemd-networkd.service
Created symlink /etc/systemd/system/dbus-org.freedesktop.network1.service →
/lib/systemd/system/systemd-networkd.service.
Created symlink
/etc/systemd/system/multi-user.target.wants/systemd-networkd.service →
/lib/systemd/system/systemd-networkd.service.
Created symlink
/etc/systemd/system/sockets.target.wants/systemd-networkd.socket →
/lib/systemd/system/systemd-networkd.socket.
Created symlink
/etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service
→ /lib/systemd/system/systemd-networkd-wait-online.service.
+ systemctl unmask systemd-resolved.service
+ systemctl enable --now systemd-resolved.service
Created symlink /etc/systemd/system/dbus-org.freedesktop.resolve1.service →
/lib/systemd/system/systemd-resolved.service.
Created symlink
/etc/systemd/system/multi-user.target.wants/systemd-resolved.service →
/lib/systemd/system/systemd-resolved.service.
+ mount -o remount,rw /sys
+ systemctl unmask systemd-udevd.service
Removed /etc/systemd/system/systemd-udevd.service.
+ systemctl start systemd-udevd.service
INFO: Preparing configuration
● systemd-networkd.service - Network Configuration
Loaded: loaded (/lib/systemd/system/systemd-networkd.service; enabled;
vendor preset: enabled)
Active: active (running) since Wed 2021-11-17 20:28:48 UTC; 770ms ago
TriggeredBy: ● systemd-networkd.socket
Docs: man:systemd-networkd.service(8)
Main PID: 1310 (systemd-network)
Status: "Processing requests..."
Tasks: 1 (limit: 9657)
Memory: 4.5M
CPU: 60ms
CGroup: /system.slice/systemd-networkd.service
└─1310 /lib/systemd/systemd-networkd
Nov 17 20:28:48 ci-321-dd1f0929 systemd[1]: Starting Network Configuration...
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Link UP
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Gained carrier
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: lo: Link UP
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: lo: Gained carrier
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: eth0: Gained IPv6LL
Nov 17 20:28:48 ci-321-dd1f0929 systemd-networkd[1310]: Enumeration completed
Nov 17 20:28:48 ci-321-dd1f0929 systemd[1]: Started Network Configuration.
Removed
/etc/systemd/system/network-online.target.wants/systemd-networkd-wait-online.service.
Removed /etc/systemd/system/sockets.target.wants/systemd-networkd.socket.
Removed /etc/systemd/system/multi-user.target.wants/systemd-networkd.service.
Removed /etc/systemd/system/dbus-org.freedesktop.network1.service.
Created symlink
/etc/systemd/system/basic.target.requires/cloud-init-dummy.service →
/etc/systemd/system/cloud-init-dummy.service.
INFO: Configuration written, rebooting...
bash: line 1: 1243 Killed bash -ec
'./debian/tests/prepare-testbed.sh && ./debian/tests/cloud-init.sh' 2> >(tee -a
/tmp/autopkgtest-lxc.qs43ksy_/downtmp/cloud-init-stderr >&2) > >(tee -a
/tmp/autopkgtest-lxc.qs43ksy_/downtmp/cloud-init-stdout)
autopkgtest [20:28:49]: test process requested reboot with marker config
<VirtSubproc>: failure: timed out waiting for container ci-321-dd1f0929 to
start; last runlevel "unknown
"
autopkgtest [20:31:00]: ERROR: testbed failure: cannot send to testbed: [Errno
32] Broken pipe
Paul
https://ci.debian.net/packages/n/netplan.io/testing/ppc64el/
diff -r pass/autostart-packages fail/autostart-packages
1c1
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
5,6c5,6
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
10c10
< python3 3.9.2-3
---
> python3 3.9.7-1
13,14c13,14
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/bonds-packages fail/bonds-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/bridges-packages fail/bridges-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/cloud-init-packages fail/cloud-init-packages
1c1
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
5,6c5,6
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
10c10
< python3 3.9.2-3
---
> python3 3.9.7-1
13,14c13,14
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/duration fail/duration
1c1
< 11745
---
> 1001
diff -r pass/ethernets-packages fail/ethernets-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/exitcode fail/exitcode
1c1
< 2
---
> 16
diff -r pass/ovs-packages fail/ovs-packages
11c11
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
30,31c30,31
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
44c44
< python3 3.9.2-3
---
> python3 3.9.7-1
49,50c49,50
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/regressions-packages fail/regressions-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/routing-packages fail/routing-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/scenarios-packages fail/scenarios-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/summary fail/summary
1,2c1,2
< ovs FLAKY timed out
< ethernets PASS
---
> ovs FLAKY non-zero exit status 1
> ethernets FAIL non-zero exit status 1
4,10c4,10
< bonds PASS
< routing PASS
< vlans PASS
< wifi PASS
< tunnels PASS
< scenarios PASS
< regressions PASS
---
> bonds FAIL non-zero exit status 1
> routing FAIL non-zero exit status 1
> vlans FLAKY non-zero exit status 1
> wifi FLAKY non-zero exit status 1
> tunnels FAIL non-zero exit status 1
> scenarios FLAKY non-zero exit status 1
> regressions FAIL non-zero exit status 1
12c12
< cloud-init PASS
---
> testbed failure: cannot send to testbed: [Errno 32] Broken pipe
diff -r pass/testbed-packages fail/testbed-packages
6c6
< bash 5.1-3+b2
---
> bash 5.1-3.1
13c13
< coreutils 8.32-4
---
> coreutils 8.32-4.1
51c51
< iproute2 5.15.0-1
---
> iproute2 5.14.0-1
126c126
< libpam-systemd 249.5-2
---
> libpam-systemd 249.6-1
132c132
< libpython3.9-minimal 3.9.7-4
---
> libpython3.9-minimal 3.9.8-1
139,140c139,140
< libsemanage-common 3.1-1
< libsemanage1 3.1-1+b2
---
> libsemanage-common 3.1-2
> libsemanage1 3.1-2
147c147
< libsystemd0 249.5-2
---
> libsystemd0 249.6-1
152c152
< libudev1 249.5-2
---
> libudev1 249.6-1
160c160
< login 1:4.8.1-1
---
> login 1:4.8.1-1.1
170,172c170,172
< openssh-client 1:8.4p1-6
< openssh-server 1:8.4p1-6
< openssh-sftp-server 1:8.4p1-6
---
> openssh-client 1:8.7p1-1
> openssh-server 1:8.7p1-1
> openssh-sftp-server 1:8.7p1-1
174c174
< passwd 1:4.8.1-1
---
> passwd 1:4.8.1-1.1
181,182c181,182
< python3-minimal 3.9.2-3
< python3.9-minimal 3.9.7-4
---
> python3-minimal 3.9.7-1
> python3.9-minimal 3.9.8-1
187,188c187,188
< systemd 249.5-2
< systemd-sysv 249.5-2
---
> systemd 249.6-1
> systemd-sysv 249.6-1
diff -r pass/testinfo.json fail/testinfo.json
2c2
< "virt_server": "autopkgtest-virt-lxc --sudo --name ci-310-f38c586c
autopkgtest-testing-ppc64el",
---
> "virt_server": "autopkgtest-virt-lxc --sudo --name ci-316-152ee3eb
> autopkgtest-testing-ppc64el",
diff -r pass/trigger fail/trigger
1c1
< iproute2%2F5.15.0-1
---
> systemd%2F249.6-1
diff -r pass/tunnels-packages fail/tunnels-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/vlans-packages fail/vlans-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/wifi-packages fail/wifi-packages
9c9
< libglib2.0-0 2.70.0-3
---
> libglib2.0-0 2.70.1-1
28,29c28,29
< libpython3-stdlib 3.9.2-3
< libpython3.9-stdlib 3.9.7-4
---
> libpython3-stdlib 3.9.7-1
> libpython3.9-stdlib 3.9.8-1
39c39
< python3 3.9.2-3
---
> python3 3.9.7-1
43,44c43,44
< python3.9 3.9.7-4
< udev 249.5-2
---
> python3.9 3.9.8-1
> udev 249.6-1
diff -r pass/worker fail/worker
1c1
< ci-worker-ppc64el-02
---
> ci-worker-ppc64el-01
--- End Message ---
--- Begin Message ---
Version: 0.103-4
After lots of experimentation and trying to replicate the DebCI locally,
it looks like I might have found the cause for this failure.
The netplan.io 0.103-4 contains a distro patch to avoid OVS (i.e. the
ovs-vsctl command) to time out if no OpenVSwitch is installed on the LXC
host, which isn't the case on DebCI.
I've also forwarded that patch upstream and got it reviewed and merged:
https://github.com/canonical/netplan/pull/266
Cheers,
Lukas
--- End Message ---
