Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
Control: tag -1 help On Thu, 2021-03-04 at 09:53 +0900, Ryutaroh Matsumoto wrote: > I also see a problem in grub-pc/sid. > Could you try > > # debci setup -f -s bullseye -a amd64 -b qemu > Then > > rrs@lenovo:~$ autopkgtest -B -U -u debci -o /tmp/multipath-tools- > > qemu > > multipath-tools -- qemu -d --show-boot -c 1 --ram-size=3072 > > /var/lib/debci/qemu/bullseye-amd64.img > (sid replaced by bullseye). I was finally able to boot and test a suggestion from Sebastien. But the testcase still fails. And, by now, you must have realized that I'm complete noob in debci/autopkgtest so I'll have to rely on some other expert in resolving this. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
> rrs@lenovo:~$ autopkgtest -B -U -u debci -o /tmp/multipath-tools-qemu > multipath-tools -- qemu -d --show-boot -c 1 --ram-size=3072 > /var/lib/debci/qemu/sid-amd64.img This should work... You could try the same command by root, but it should work under an ordinary user. > Booting from Hard Disk... > autopkgtest-virt-qemu: DBG: cleanup... > : failure: timed out waiting for "login prompt on ttyS0" > autopkgtest [20:12:41]: ERROR: testbed failure: cannot send to testbed: > [Errno 32] Broken pipe Have you seen "Grub" or "Linux"? If not, the installation of grub and/or linux-image-amd64 is broken in the VM image /var/lib/debci/qemu/sid-amd64.img. I wonder if /var/lib/debci/qemu/sid-amd64.img can be booted under virt-manager. I also see a problem in grub-pc/sid. Could you try # debci setup -f -s bullseye -a amd64 -b qemu Then > rrs@lenovo:~$ autopkgtest -B -U -u debci -o /tmp/multipath-tools-qemu > multipath-tools -- qemu -d --show-boot -c 1 --ram-size=3072 > /var/lib/debci/qemu/bullseye-amd64.img (sid replaced by bullseye). Best regards, Ryutaroh
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
Hello Sebastien On Wed, 2021-03-03 at 14:49 +0100, Sebastien Bacher wrote: > It's similar to > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981231 > and a warning from the new gdisks version which is printed on stderr, > or > autopkgtest fails on stderr content by default > > Using Restrictions: allow-stderr should workaround the issue > That did not help. One had it already and I added the same to the other test. ``` Tests: kpartx-file-loopback Depends: kpartx, qemu-utils, gdisk Restrictions: needs-root, isolation-machine, allow-stderr Tests: tgtbasedmpaths Depends: multipath-tools, tgt, open-iscsi, fio, lsscsi Restrictions: needs-root, isolation-machine, allow-stderr ``` But it still fails. ``` rrs@lenovo:/tmp/multipath-tools-qemu$ cat summary kpartx-file-loopback PASS tgtbasedmpaths FAIL non-zero exit status 1 21:07 ♒♒♒ ☺ rrs@lenovo:/tmp/multipath-tools-qemu$ tail -n 50 log Mar 03 15:33:13 | sdd: checker timeout = 30 s (setting: kernel sysfs) Mar 03 15:33:13 | sdd: tur state = up = paths list = uuid hcildev dev_t pri dm_st chk_st vend/prod/revdev_st 2:0:0:1 sda 8:0 -1 undef undef IET,VIRTUAL-DISK unknown 3:0:0:1 sdb 8:16 -1 undef undef IET,VIRTUAL-DISK unknown 4:0:0:1 sdc 8:32 -1 undef undef IET,VIRTUAL-DISK unknown 5:0:0:1 sdd 8:48 -1 undef undef IET,VIRTUAL-DISK unknown No devices found Test WWN should now point to DM Mar 03 15:33:13 | libdevmapper version 1.02.175 (2021-01-08) Mar 03 15:33:13 | DM multipath kernel driver v1.14.0 Mar 03 15:33:13 | unloading const prioritizer Mar 03 15:33:13 | unloading tur checker + dmsetup table + echo Test WWN should now point to DM + grep dm + readlink /dev/disk/by-id/scsi-36e010001 autopkgtest [21:03:14]: test tgtbasedmpaths: ---] autopkgtest-virt-qemu: DBG: executing copyup /tmp/autopkgtest.6dFWDl/tgtbasedmpaths-stdout /tmp/multipath-tools- qemu/tgtbasedmpaths-stdout autopkgtest-virt-qemu: DBG: ['cmdls', "(['/tmp/autopkgtest- qemu.0ejkv6rf/runcmd', 'sh', '-ec', 'cat ", 'deststdout', "<_io.BufferedWriter name='/tmp/multipath-tools-qemu/tgtbasedmpaths-stdout'>", 'devnull_read', <_io.BufferedReader name='/] autopkgtest-virt-qemu: DBG: +< /tmp/autopkgtest-qemu.0ejkv6rf/runcmd sh -ec cat cat autopkgtest-virt-qemu: DBG: +>? autopkgtest-virt-qemu: DBG: +", 'deststdout', "<_io.BufferedWriter name='/tmp/multipath-tools-qemu/tgtbasedmpaths-stderr'>", 'devnull_read', <_io.BufferedReader name='/] autopkgtest-virt-qemu: DBG: +< /tmp/autopkgtest-qemu.0ejkv6rf/runcmd sh -ec cat cat autopkgtest-virt-qemu: DBG: +>? autopkgtest-virt-qemu: DBG: +", 'deststdout', "<_io.BufferedReader name='/dev/null'>", 'devnull_read', <_io.BufferedReader name='/dev/null'>] autopkgtest-virt-qemu: DBG: +< /tmp/autopkgtest-qemu.0ejkv6rf/runcmd sh -ec cd /tmp/autopkgtest.6dFWDl/tgtbasedmpaths-artifacts/; tar -- warning=none -c . -f - autopkgtest-virt-qemu: DBG: +> tar --directory /tmp/multipath-tools- qemu/artifacts/ --warning=none --preserve-permissions --extract --no- same-owner -f - autopkgtest-virt-qemu: DBG: +>? autopkgtest-virt-qemu: DBG: +http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
On Wed, 2021-03-03 at 20:14 +0530, Ritesh Raj Sarraf wrote: > Is it something you can help me with, to reproduce the actual > multipath-tools issue ? I figured out the setup for debci/autopkgtest all thanks to your bug reports that motivated me. :-) -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
On Wed, 2021-02-24 at 08:50 +0900, Ryutaroh Matsumoto wrote: > > Other than the tests, does multipath-tools work proper for you ? > > From the logs you've shared, multipath does report to see the iSCSI > > LUNs. Now whether a proper map was created or not, is not clear > > from > > the logs. > > I have seen no problems in bin. packages from src:multipath-tools. > I run autopkgtest qemu on all packages with standard priority and > in task-gnome-desktop, and reported all the errors I observed (as > tests finished) except the packages already erred on ci.debian.net. > Some of them turned out to be real errors... I tried to reproduce it with the setup you mentioned. ``` rrs@lenovo:~$ autopkgtest -B -U -u debci -o /tmp/multipath-tools-qemu multipath-tools -- qemu -d --show-boot -c 1 --ram-size=3072 /var/lib/debci/qemu/sid-amd64.img autopkgtest [20:11:41]: starting date: 2021-03-03 autopkgtest [20:11:41]: version 5.16 autopkgtest [20:11:41]: host lenovo; command line: /usr/bin/autopkgtest -B -U -u debci -o /tmp/multipath-tools-qemu multipath-tools -- qemu -d --show-boot -c 1 --ram-size=3072 /var/lib/debci/qemu/sid-amd64.img autopkgtest-virt-qemu: DBG: executing open autopkgtest-virt-qemu: DBG: find_free_port: trying 10022 autopkgtest-virt-qemu: DBG: find_free_port: 10022 is free autopkgtest-virt-qemu: DBG: execute-timeout: qemu-img info -- output=json /var/lib/debci/qemu/sid-amd64.img autopkgtest-virt-qemu: DBG: Creating temporary overlay image in /tmp/autopkgtest-qemu.3tdfx4at/overlay.img autopkgtest-virt-qemu: DBG: execute-timeout: qemu-img create -f qcow2 - F qcow2 -b /var/lib/debci/qemu/sid-amd64.img /tmp/autopkgtest- qemu.3tdfx4at/overlay.img autopkgtest-virt-qemu: DBG: Forwarding local port 10022 to VM ssh port 22 autopkgtest-virt-qemu: DBG: Detected KVM capable Intel host CPU, enabling nested KVM autopkgtest-virt-qemu: DBG: expect: " login: " ``` and ``` SeaBIOS (version 1.14.0-2) iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+BFF8F2D0+BFECF2D0 CA00 Booting from Hard Disk... autopkgtest-virt-qemu: DBG: cleanup... : failure: timed out waiting for "login prompt on ttyS0" autopkgtest [20:12:41]: ERROR: testbed failure: cannot send to testbed: [Errno 32] Broken pipe 20:12 ♒♒♒☹ => 16 ``` Is it something you can help me with, to reproduce the actual multipath-tools issue ? -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
It's similar to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981231 and a warning from the new gdisks version which is printed on stderr, or autopkgtest fails on stderr content by default Using Restrictions: allow-stderr should workaround the issue
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
> Other than the tests, does multipath-tools work proper for you ? > From the logs you've shared, multipath does report to see the iSCSI > LUNs. Now whether a proper map was created or not, is not clear from > the logs. I have seen no problems in bin. packages from src:multipath-tools. I run autopkgtest qemu on all packages with standard priority and in task-gnome-desktop, and reported all the errors I observed (as tests finished) except the packages already erred on ci.debian.net. Some of them turned out to be real errors... Best regards, Ryutaroh
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
On Tue, 2021-02-23 at 18:48 +0900, Ryutaroh Matsumoto wrote: > Hi Ritesh Raj Sarraf, > Thank you for paying your attention to this. > > > I skimmed into the logs but I'm not sure what failure is being > > referred > > to here. > > I meant the below part in "log" file. Feel free to downgrade the > severity > if you consider this as a false positive. > In the following, the actual test results seem different from what > are > expected by the test script in multipath-tools. I really don't have much insight into these tests. So I can't judge if the report is correct or not. I usually do manual tests as a pre-requisite to my uploads. Looking at the git logs, these tests were contributed by the Ubuntu folks (CCed). Hopefully they may have more insight into it. Other than the tests, does multipath-tools work proper for you ? From the logs you've shared, multipath does report to see the iSCSI LUNs. Now whether a proper map was created or not, is not clear from the logs. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
Hi Ritesh Raj Sarraf, Thank you for paying your attention to this. > I skimmed into the logs but I'm not sure what failure is being referred > to here. I meant the below part in "log" file. Feel free to downgrade the severity if you consider this as a false positive. In the following, the actual test results seem different from what are expected by the test script in multipath-tools. NVMe module may not be loaded = paths list = uuid hcildev dev_t pri dm_st chk_st vend/prod/revdev_st 2:0:0:1 sda 8:0 -1 undef undef IET,VIRTUAL-DISK unknown 3:0:0:1 sdb 8:16 -1 undef undef IET,VIRTUAL-DISK unknown 4:0:0:1 sdc 8:32 -1 undef undef IET,VIRTUAL-DISK unknown 5:0:0:1 sdd 8:48 -1 undef undef IET,VIRTUAL-DISK unknown No devices found Test WWN should now point to DM + lsscsi -liv list_ndevices: scandir: /sys/class/nvme/: No such file or directory + multipath -v3 -ll Feb 23 00:09:07 | set open fds limit to 1048576/1048576 Feb 23 00:09:07 | loading //lib/multipath/libchecktur.so checker Feb 23 00:09:07 | checker tur: message table size = 3 Feb 23 00:09:07 | loading //lib/multipath/libprioconst.so prioritizer Feb 23 00:09:07 | _init_foreign: foreign library "nvme" is not enabled Feb 23 00:09:07 | sr0: device node name blacklisted Feb 23 00:09:07 | vda: device node name blacklisted Feb 23 00:09:07 | fd0: device node name blacklisted Feb 23 00:09:07 | sda: size = 204800 Feb 23 00:09:07 | sda: vendor = IET Feb 23 00:09:07 | sda: product = VIRTUAL-DISK Feb 23 00:09:07 | sda: rev = 0001 Feb 23 00:09:07 | sda: h:b:t:l = 2:0:0:1 Feb 23 00:09:07 | sda: tgt_node_name = iqn.2016-11.foo.com:target.iscsi Feb 23 00:09:07 | sda: 1024 cyl, 4 heads, 50 sectors/track, start at 0 Feb 23 00:09:07 | sda: vpd_vendor_id = 0 "undef" (setting: multipath internal) Feb 23 00:09:07 | sda: serial = beaf11 Feb 23 00:09:07 | sda: detect_checker = yes (setting: multipath internal) Feb 23 00:09:07 | sda: path_checker = tur (setting: multipath internal) Feb 23 00:09:07 | sda: checker timeout = 30 s (setting: kernel sysfs) Feb 23 00:09:07 | sda: tur state = up Feb 23 00:09:07 | sdb: size = 204800 Feb 23 00:09:07 | sdb: vendor = IET Feb 23 00:09:07 | sdb: product = VIRTUAL-DISK Feb 23 00:09:07 | sdb: rev = 0001 Feb 23 00:09:07 | sdb: h:b:t:l = 3:0:0:1 Feb 23 00:09:07 | sdb: tgt_node_name = iqn.2016-11.foo.com:target.iscsi Feb 23 00:09:07 | sdb: 1024 cyl, 4 heads, 50 sectors/track, start at 0 Feb 23 00:09:07 | sdb: vpd_vendor_id = 0 "undef" (setting: multipath internal) Feb 23 00:09:07 | sdb: serial = beaf11 Feb 23 00:09:07 | sdb: detect_checker = yes (setting: multipath internal) Feb 23 00:09:07 | sdb: path_checker = tur (setting: multipath internal) Feb 23 00:09:07 | sdb: checker timeout = 30 s (setting: kernel sysfs) Feb 23 00:09:07 | sdb: tur state = up Feb 23 00:09:07 | sdc: size = 204800 Feb 23 00:09:07 | sdc: vendor = IET Feb 23 00:09:07 | sdc: product = VIRTUAL-DISK Feb 23 00:09:07 | sdc: rev = 0001 Feb 23 00:09:07 | sdc: h:b:t:l = 4:0:0:1 Feb 23 00:09:07 | sdc: tgt_node_name = iqn.2016-11.foo.com:target.iscsi Feb 23 00:09:07 | sdc: 1024 cyl, 4 heads, 50 sectors/track, start at 0 Feb 23 00:09:07 | sdc: vpd_vendor_id = 0 "undef" (setting: multipath internal) Feb 23 00:09:07 | sdc: serial = beaf11 Feb 23 00:09:07 | sdc: detect_checker = yes (setting: multipath internal) Feb 23 00:09:07 | sdc: path_checker = tur (setting: multipath internal) Feb 23 00:09:07 | sdc: checker timeout = 30 s (setting: kernel sysfs) Feb 23 00:09:07 | sdc: tur state = up Feb 23 00:09:07 | sdd: size = 204800 Feb 23 00:09:07 | sdd: vendor = IET Feb 23 00:09:07 | sdd: product = VIRTUAL-DISK Feb 23 00:09:07 | sdd: rev = 0001 Feb 23 00:09:07 | sdd: h:b:t:l = 5:0:0:1 Feb 23 00:09:07 | sdd: tgt_node_name = iqn.2016-11.foo.com:target.iscsi Feb 23 00:09:07 | sdd: 1024 cyl, 4 heads, 50 sectors/track, start at 0 Feb 23 00:09:07 | sdd: vpd_vendor_id = 0 "undef" (setting: multipath internal) Feb 23 00:09:07 | sdd: serial = beaf11 Feb 23 00:09:07 | sdd: detect_checker = yes (setting: multipath internal) Feb 23 00:09:07 | sdd: path_checker = tur (setting: multipath internal) Feb 23 00:09:07 | sdd: checker timeout = 30 s (setting: kernel sysfs) Feb 23 00:09:07 | sdd: tur state = up Feb 23 00:09:07 | libdevmapper version 1.02.175 (2021-01-08) Feb 23 00:09:07 | DM multipath kernel driver v1.14.0 Feb 23 00:09:07 | unloading const prioritizer Feb 23 00:09:07 | unloading tur checker + dmsetup table + echo Test WWN should now point to DM + grep dm + readlink /dev/disk/by-id/scsi-36e010001 Best regards, Ryutaroh
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
Hi Ryutaroh Matsumoto, On Tue, 2021-02-23 at 09:22 +0900, Ryutaroh Matsumoto wrote: > I run autopkgtest -U -B -u debci mutipath-tools -- qemu with a > testbed made by > debci setup -f -s sid -a amd64 -b qemu. > > The testsuite gives > kpartx-file-loopback FAIL stderr: Warning: Partition table header > claims that the size of partition table > tgtbasedmpaths FAIL non-zero exit status 1 > > Looking at the log, failure of tgtbasedmpaths seems a real error. I skimmed into the logs but I'm not sure what failure is being referred to here. -- Ritesh Raj Sarraf | http://people.debian.org/~rrs Debian - The Universal Operating System signature.asc Description: This is a digitally signed message part
Bug#983360: multipath-tools: autopkgtest failure on the qemu testbed
Source: multipath-tools Version: 0.8.5-1 Severity: important Tags: bullseye sid Dear Maintainer, I run autopkgtest -U -B -u debci mutipath-tools -- qemu with a testbed made by debci setup -f -s sid -a amd64 -b qemu. The testsuite gives kpartx-file-loopback FAIL stderr: Warning: Partition table header claims that the size of partition table tgtbasedmpaths FAIL non-zero exit status 1 Looking at the log, failure of tgtbasedmpaths seems a real error. The log is attached. Best regards, Ryutaroh Matsumoto -- System Information: Debian Release: bullseye/sid APT prefers testing APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental') Architecture: amd64 (x86_64) Kernel: Linux 5.9.16-preempt (SMP w/12 CPU threads; PREEMPT) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en Shell: /bin/sh linked to /usr/bin/dash Init: systemd (via /run/systemd/system) LSM: AppArmor: enabled multipath-tools-autopkgtest.tar.xz Description: application/xz
