Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On Wed, Sep 11, 2019 at 02:57:50PM +0100, Richard W.M. Jones wrote: > On Wed, Sep 11, 2019 at 08:48:28AM -0500, Eric Blake wrote: > > On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > > > Very much a work in progress as there are still many tests using > > > qemu-io which are candidates for conversion. > > > > > > You'll notice at the end of test-full.sh that the new test has some > > > duplicated code which looks as if it ought to be refactored into a > > > Python function. When I tried to do that, I got loads of strange > > > Python problems which may indicate bugs in nbdsh itself or problems > > > with my understanding of how Python works, but I wasn't sure how to > > > fix it. The current implementation works. > > > > I'm not sure if I'd spot the problem, but I'd least need to see your > > alternative that didn't work before I can tell for sure :) > > Attached is my alternative (on top of the current patch), which seems > to me straightforward, but actually fails with the attached error. > > I tried variations inserting ‘global h’ in various places but couldn't > make it work. > > Also note there are two related errors, because ‘nbd’ somehow get > "unimported" from the function too. > > Needs help from someone who knows more Python :-) This turned out to be a bug in nbdsh: https://github.com/libguestfs/libnbd/commit/d6cbd130101add28431bd6e67aa2ea0430a9234e Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 100 libraries supported. http://fedoraproject.org/wiki/MinGW ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On Wed, Sep 11, 2019 at 08:48:28AM -0500, Eric Blake wrote: > On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > > Very much a work in progress as there are still many tests using > > qemu-io which are candidates for conversion. > > > > You'll notice at the end of test-full.sh that the new test has some > > duplicated code which looks as if it ought to be refactored into a > > Python function. When I tried to do that, I got loads of strange > > Python problems which may indicate bugs in nbdsh itself or problems > > with my understanding of how Python works, but I wasn't sure how to > > fix it. The current implementation works. > > I'm not sure if I'd spot the problem, but I'd least need to see your > alternative that didn't work before I can tell for sure :) Attached is my alternative (on top of the current patch), which seems to me straightforward, but actually fails with the attached error. I tried variations inserting ‘global h’ in various places but couldn't make it work. Also note there are two related errors, because ‘nbd’ somehow get "unimported" from the function too. Needs help from someone who knows more Python :-) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top >From d88957e9572e69490e109526f90574fe8589 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" Date: Wed, 11 Sep 2019 14:54:49 +0100 Subject: [PATCH] tests/test-full.sh: Refactor to tests into a single function (NOT WORKING). --- tests/test-full.sh | 24 ++-- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/tests/test-full.sh b/tests/test-full.sh index 8f237dd..f060f4f 100755 --- a/tests/test-full.sh +++ b/tests/test-full.sh @@ -55,19 +55,15 @@ nbdsh --connect "nbd+unix://?socket=$sock" \ # All writes should fail with the ENOSPC error. nbdsh --connect "nbd+unix://?socket=$sock" \ -c ' -try: -h.pwrite (bytearray (512), 0) -# This should not happen. -exit (1) -except nbd.Error as ex: -# Check the errno is expected. -assert ex.errno == "ENOSPC" +def test (offset): +try: +h.pwrite (bytearray (512), offset) +# This should not happen. +exit (1) +except nbd.Error as ex: +# Check the errno is expected. +assert ex.errno == "ENOSPC" -try: -h.pwrite (bytearray (512), 1048064) -# This should not happen. -exit (1) -except nbd.Error as ex: -# Check the errno is expected. -assert ex.errno == "ENOSPC" +test (0) +test (1048064) ' -- 2.23.0 /home/rjones/d/nbdkit/server/nbdkit -v -P full.pid -U /tmp/tmp.sHet2Luahj -- /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so 1M nbdkit: debug: TLS disabled: could not load TLS certificates nbdkit: debug: registering plugin /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so nbdkit: debug: registered plugin /home/rjones/d/nbdkit/plugins/full/.libs/nbdkit-full-plugin.so (name full) nbdkit: debug: full: load nbdkit: debug: full: config key=size, value=1M nbdkit: debug: full: config_complete nbdkit: debug: using thread model: parallel nbdkit: debug: bound to unix socket /tmp/tmp.sHet2Luahj nbdkit: debug: forked into background (new pid = 422301) nbdkit: debug: written pidfile /home/rjones/d/nbdkit/tests/full.pid libnbd: debug: nbd1: nbd_create: opening handle libnbd: debug: nbd1: nbd_close: closing handle libnbd: debug: nbd2: nbd_create: opening handle libnbd: debug: nbd2: nbd_close: closing handle libnbd: debug: nbd3: nbd_create: opening handle libnbd: debug: nbd3: nbd_close: closing handle libnbd: debug: nbd4: nbd_create: opening handle libnbd: debug: nbd4: nbd_close: closing handle libnbd: debug: nbd5: nbd_create: opening handle libnbd: debug: nbd5: nbd_connect_uri: enter: uri="nbd+unix://?socket=/tmp/tmp.sHet2Luahj" libnbd: debug: nbd5: nbd_connect_uri: event CmdConnectUnix: START -> CONNECT_UNIX.START libnbd: debug: nbd5: nbd_connect_uri: transition: CONNECT_UNIX.START -> CONNECT.START libnbd: debug: nbd5: nbd_connect_uri: poll start: events=4 libnbd: debug: nbd5: nbd_connect_uri: poll end: r=1 revents=4 libnbd: debug: nbd5: nbd_connect_uri: event NotifyWrite: CONNECT.START -> CONNECT.CONNECTING libnbd: debug: nbd5: nbd_connect_uri: transition: CONNECT.CONNECTING -> MAGIC.START libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.START -> MAGIC.RECV_MAGIC libnbd: debug: nbd5: nbd_connect_uri: poll start: events=1 nbdkit: debug: accepted connection nbdkit: full[1]: debug: newstyle negotiation: flags: global 0x3 libnbd: debug: nbd5: nbd_connect_uri: poll end: r=1 revents=1 libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.RECV_MAGIC -> MAGIC.CHECK_MAGIC libnbd: debug: nbd5: nbd_connect_uri: transition: MAGIC.CHECK_MAGIC -> NEWSTYLE.START libnbd:
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > nbdsh has some advantages over qemu-io: > > - scriptable > > - allows us to more finely control NBD commands, such as >making subsector-sized requests and controlling how >many commands are sent on the wire > > - can write controlled patterns > > - can read NBD export flags - can more easily get at exact errno response returned by server > # Because error rate is 0%, reads should never fail. > -qemu-io -r -f raw "nbd+unix://?socket=$sock" \ > --c "r 0M 10M" \ > --c "r 20M 10M" \ > --c "r 40M 10M" \ > --c "r 60M 10M" > +nbdsh --connect "nbd+unix://?socket=$sock" \ > + -c 'mbytes = 2**20' \ > + -c 'h.pread(10*mbytes, 0)' \ > + -c 'h.pread(10*mbytes, 20*mbytes)' \ > + -c 'h.pread(10*mbytes, 40*mbytes)' \ > + -c 'h.pread(10*mbytes, 60*mbytes)' A bit annoying that qemu-io and libnbd picked opposite ordering for length vs. offset, but such is life. LGTM. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
Re: [Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
On 9/11/19 5:40 AM, Richard W.M. Jones wrote: > Very much a work in progress as there are still many tests using > qemu-io which are candidates for conversion. > > You'll notice at the end of test-full.sh that the new test has some > duplicated code which looks as if it ought to be refactored into a > Python function. When I tried to do that, I got loads of strange > Python problems which may indicate bugs in nbdsh itself or problems > with my understanding of how Python works, but I wasn't sure how to > fix it. The current implementation works. I'm not sure if I'd spot the problem, but I'd least need to see your alternative that didn't work before I can tell for sure :) -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
nbdsh has some advantages over qemu-io:
- scriptable
- allows us to more finely control NBD commands, such as
making subsector-sized requests and controlling how
many commands are sent on the wire
- can write controlled patterns
- can read NBD export flags
---
README | 2 ++
tests/Makefile.am | 5 +++--
tests/test-error0.sh | 13 +++--
tests/test-error100.sh | 19 ---
tests/test-full.sh | 32 +++-
5 files changed, 47 insertions(+), 24 deletions(-)
diff --git a/README b/README
index b78f490..187da49 100644
--- a/README
+++ b/README
@@ -164,6 +164,8 @@ For non-essential enhancements to the test suite:
- mke2fs (from e2fsprogs)
+ - nbdsh (from libnbd)
+
- qemu-img, qemu-io, qemu-nbd (usually shipped with qemu)
- sfdisk (from util-linux)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b581cf6..b5806bb 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -355,12 +355,13 @@ if HAVE_LIBGUESTFS
check_PROGRAMS += $(LIBGUESTFS_TESTS)
TESTS += $(LIBGUESTFS_TESTS)
-# Use the 'direct' backend, and ensure maximum libguestfs debugging is
-# written to the *.log files in case there is a problem.
+# Use the 'direct' backend, and ensure maximum libguestfs and libnbd
+# debugging is written to the *.log files in case there is a problem.
TESTS_ENVIRONMENT += \
LIBGUESTFS_ATTACH_METHOD=appliance \
LIBGUESTFS_DEBUG=1 \
LIBGUESTFS_TRACE=1 \
+ LIBNBD_DEBUG=1 \
$(NULL)
# Common test library.
diff --git a/tests/test-error0.sh b/tests/test-error0.sh
index fe97354..de9dca2 100755
--- a/tests/test-error0.sh
+++ b/tests/test-error0.sh
@@ -34,7 +34,7 @@ source ./functions.sh
set -e
set -x
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="$sock error0.pid"
@@ -47,8 +47,9 @@ start_nbdkit -P error0.pid -U $sock \
pattern 1G error-rate=0%
# Because error rate is 0%, reads should never fail.
-qemu-io -r -f raw "nbd+unix://?socket=$sock" \
--c "r 0M 10M" \
--c "r 20M 10M" \
--c "r 40M 10M" \
--c "r 60M 10M"
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c 'mbytes = 2**20' \
+ -c 'h.pread(10*mbytes, 0)' \
+ -c 'h.pread(10*mbytes, 20*mbytes)' \
+ -c 'h.pread(10*mbytes, 40*mbytes)' \
+ -c 'h.pread(10*mbytes, 60*mbytes)'
diff --git a/tests/test-error100.sh b/tests/test-error100.sh
index 64056ce..a2d3bdc 100755
--- a/tests/test-error100.sh
+++ b/tests/test-error100.sh
@@ -34,7 +34,7 @@ source ./functions.sh
set -e
set -x
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="$sock error100.pid"
@@ -46,11 +46,16 @@ start_nbdkit -P error100.pid -U $sock \
--filter=error \
pattern 1G error-rate=100%
-# The error rate is 100% so every operation must fail.
+# The error rate is 100% so every operation must fail with error EIO.
for i in {1..100}; do
-if qemu-io -r -f raw "nbd+unix://?socket=$sock" \
- -c "r 0 512"; then
-echo "$0: expected qemu-io command to fail"
-exit 1
-fi
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c '
+try:
+h.pread (512, 0)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "EIO"
+'
done
diff --git a/tests/test-full.sh b/tests/test-full.sh
index 47bff5e..8f237dd 100755
--- a/tests/test-full.sh
+++ b/tests/test-full.sh
@@ -36,7 +36,7 @@
source ./functions.sh
set -e
-requires qemu-io --version
+requires nbdsh --version
sock=`mktemp -u`
files="full.pid $sock full.out"
@@ -47,13 +47,27 @@ cleanup_fn rm -f $files
start_nbdkit -P full.pid -U $sock full 1M
# All reads should succeed.
-qemu-io -f raw "nbd+unix://?socket=$sock" \
--c 'r -v 0 512' \
--c 'r -v 512 512' \
--c 'r -v 1048064 512'
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c 'h.pread (512, 0)' \
+ -c 'h.pread (512, 512)' \
+ -c 'h.pread (512, 1048064)'
# All writes should fail with the ENOSPC error.
-! LANG=C qemu-io -f raw "nbd+unix://?socket=$sock" \
--c 'w -P 1 0 512' \
--c 'w -P 2 1048064 512' >& full.out
-grep "No space left on device" full.out
+nbdsh --connect "nbd+unix://?socket=$sock" \
+ -c '
+try:
+h.pwrite (bytearray (512), 0)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "ENOSPC"
+
+try:
+h.pwrite (bytearray (512), 1048064)
+# This should not happen.
+exit (1)
+except nbd.Error as ex:
+# Check the errno is expected.
+assert ex.errno == "ENOSPC"
+'
--
2.23.0
___
Libguestfs mailing list
Libguestfs@redhat.com
https://www.redhat.com/mailman/listinfo/libguestfs
[Libguestfs] [PATCH nbdkit] tests: Convert some tests to use nbdsh instead of qemu-io.
Very much a work in progress as there are still many tests using qemu-io which are candidates for conversion. You'll notice at the end of test-full.sh that the new test has some duplicated code which looks as if it ought to be refactored into a Python function. When I tried to do that, I got loads of strange Python problems which may indicate bugs in nbdsh itself or problems with my understanding of how Python works, but I wasn't sure how to fix it. The current implementation works. Rich. ___ Libguestfs mailing list Libguestfs@redhat.com https://www.redhat.com/mailman/listinfo/libguestfs
