Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 12/11/23 15:32, Andrey Drobyshev wrote: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > > Signed-off-by: Andrey Drobyshev > --- > tests/qemu-iotests/271 | 1 + > tests/qemu-iotests/308 | 2 ++ > tests/qemu-iotests/common.rc | 7 +++ > tests/qemu-iotests/tests/file-io-error | 1 + > 4 files changed, 11 insertions(+) > > diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 > index 59a6fafa2f..1424b6954d 100755 > --- a/tests/qemu-iotests/271 > +++ b/tests/qemu-iotests/271 > @@ -44,6 +44,7 @@ _supported_fmt qcow2 > _supported_proto file nfs > _supported_os Linux > _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file > refcount_bits=1[^0-9] > +_require_no_o_direct > > l2_offset=$((0x4)) > > diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 > index de12b2b1b9..535455e5b1 100755 > --- a/tests/qemu-iotests/308 > +++ b/tests/qemu-iotests/308 > @@ -52,6 +52,8 @@ _unsupported_fmt vpc > _supported_proto file # We create the FUSE export manually > _supported_os Linux # We need /dev/urandom > > +_require_no_o_direct > + > # $1: Export ID > # $2: Options (beyond the node-name and ID) > # $3: Expected return value (defaults to 'return') > diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc > index 95c12577dd..f61eae73b4 100644 > --- a/tests/qemu-iotests/common.rc > +++ b/tests/qemu-iotests/common.rc > @@ -857,6 +857,13 @@ _check_o_direct() > [[ "$out" != *"O_DIRECT"* ]] > } > > +_require_no_o_direct() > +{ > +if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then > +_notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)" > +fi > +} > + > _require_o_direct() > { > if ! _check_o_direct; then > diff --git a/tests/qemu-iotests/tests/file-io-error > b/tests/qemu-iotests/tests/file-io-error > index 88ee5f670c..2b8dc7f009 100755 > --- a/tests/qemu-iotests/tests/file-io-error > +++ b/tests/qemu-iotests/tests/file-io-error > @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 > # Format-agnostic (we do not use any), but we do test the file protocol > _supported_proto file > _require_drivers blkdebug null-co > +_require_no_o_direct > > if [ "$IMGOPTSSYNTAX" = "true" ]; then > # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts, Ping
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 1/11/24 14:53, Andrey Drobyshev wrote: > On 12/11/23 15:32, Andrey Drobyshev wrote: >> There're tests whose logic implies running without O_DIRECT set, >> otherwise they fail when running iotests in '-nocache' mode. For these >> tests let's add _require_no_o_direct() helper which can be put in the >> preabmle and which makes sure '-nocache' isn't set. Use it to skip >> running the following tests: >> >> * 271: creates files with unaligned sizes, thus producing multiple >> errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' >> permission without 'resize': Image size is not a multiple of request >> alignment >> >> * 308, file-io-error: use fuse exports. Though fuse does have >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) >> we aren't using it yet, thus getting errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT >> >> Signed-off-by: Andrey Drobyshev >> --- >> tests/qemu-iotests/271 | 1 + >> tests/qemu-iotests/308 | 2 ++ >> tests/qemu-iotests/common.rc | 7 +++ >> tests/qemu-iotests/tests/file-io-error | 1 + >> 4 files changed, 11 insertions(+) >> >> diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271 >> index 59a6fafa2f..1424b6954d 100755 >> --- a/tests/qemu-iotests/271 >> +++ b/tests/qemu-iotests/271 >> @@ -44,6 +44,7 @@ _supported_fmt qcow2 >> _supported_proto file nfs >> _supported_os Linux >> _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file >> refcount_bits=1[^0-9] >> +_require_no_o_direct >> >> l2_offset=$((0x4)) >> >> diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 >> index de12b2b1b9..535455e5b1 100755 >> --- a/tests/qemu-iotests/308 >> +++ b/tests/qemu-iotests/308 >> @@ -52,6 +52,8 @@ _unsupported_fmt vpc >> _supported_proto file # We create the FUSE export manually >> _supported_os Linux # We need /dev/urandom >> >> +_require_no_o_direct >> + >> # $1: Export ID >> # $2: Options (beyond the node-name and ID) >> # $3: Expected return value (defaults to 'return') >> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc >> index 95c12577dd..f61eae73b4 100644 >> --- a/tests/qemu-iotests/common.rc >> +++ b/tests/qemu-iotests/common.rc >> @@ -857,6 +857,13 @@ _check_o_direct() >> [[ "$out" != *"O_DIRECT"* ]] >> } >> >> +_require_no_o_direct() >> +{ >> +if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then >> +_notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)" >> +fi >> +} >> + >> _require_o_direct() >> { >> if ! _check_o_direct; then >> diff --git a/tests/qemu-iotests/tests/file-io-error >> b/tests/qemu-iotests/tests/file-io-error >> index 88ee5f670c..2b8dc7f009 100755 >> --- a/tests/qemu-iotests/tests/file-io-error >> +++ b/tests/qemu-iotests/tests/file-io-error >> @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 >> # Format-agnostic (we do not use any), but we do test the file protocol >> _supported_proto file >> _require_drivers blkdebug null-co >> +_require_no_o_direct >> >> if [ "$IMGOPTSSYNTAX" = "true" ]; then >> # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts, > > Ping Ping
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On Mon, Dec 11, 2023 at 03:32:23PM +0200, Andrey Drobyshev wrote: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip preamble > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment I wonder if we can instead tweak the test to use larger sizes such that all accesses ARE aligned even with O_DIRECT. /me goes and reads the test... # Note that the image size is not a multiple of the cluster size _reset_img 2083k Ah - we really DO have a test that depends on odd sizing; where changing it to be more nicely aligned will break other assumptions in the test. > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT And I agree that this one is beyond our control, so adding skip support makes sense. > > Signed-off-by: Andrey Drobyshev > --- > tests/qemu-iotests/271 | 1 + > tests/qemu-iotests/308 | 2 ++ > tests/qemu-iotests/common.rc | 7 +++ > tests/qemu-iotests/tests/file-io-error | 1 + > 4 files changed, 11 insertions(+) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > > Signed-off-by: Andrey Drobyshev How are you running qemu-iotests to make these tests fail? I tried to reproduce, but they just pass for me: $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error [...] 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) Passed all 3 iotests $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error 271 not run[11:21:20] [11:21:21] ... not suitable for this image format: raw 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) Not run: 271 Passed all 2 iotests Kevin
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 1/26/24 12:24, Kevin Wolf wrote: > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: >> There're tests whose logic implies running without O_DIRECT set, >> otherwise they fail when running iotests in '-nocache' mode. For these >> tests let's add _require_no_o_direct() helper which can be put in the >> preabmle and which makes sure '-nocache' isn't set. Use it to skip >> running the following tests: >> >> * 271: creates files with unaligned sizes, thus producing multiple >> errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' >> permission without 'resize': Image size is not a multiple of request >> alignment >> >> * 308, file-io-error: use fuse exports. Though fuse does have >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) >> we aren't using it yet, thus getting errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT >> >> Signed-off-by: Andrey Drobyshev > > How are you running qemu-iotests to make these tests fail? I tried to > reproduce, but they just pass for me: > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > [...] > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > Passed all 3 iotests > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > 271 not run[11:21:20] [11:21:21] ... not > suitable for this image format: raw > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > Not run: 271 > Passed all 2 iotests > > Kevin > As for the test 271, I imagine this might be caused by different request alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If I print the alignment out explicitly, I get: qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' permission without 'resize': Image size is not a multiple of request alignment: 4096 For the record, I'm running tests on ext4. I'm not sure about the fuse tests though. Could it also have smth to do with the underlying fs? Andrey
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
Am 26.01.2024 um 12:04 hat Andrey Drobyshev geschrieben: > On 1/26/24 12:24, Kevin Wolf wrote: > > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > >> There're tests whose logic implies running without O_DIRECT set, > >> otherwise they fail when running iotests in '-nocache' mode. For these > >> tests let's add _require_no_o_direct() helper which can be put in the > >> preabmle and which makes sure '-nocache' isn't set. Use it to skip > >> running the following tests: > >> > >> * 271: creates files with unaligned sizes, thus producing multiple > >> errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > >> permission without 'resize': Image size is not a multiple of request > >> alignment > >> > >> * 308, file-io-error: use fuse exports. Though fuse does have > >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > >> we aren't using it yet, thus getting errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > >> > >> Signed-off-by: Andrey Drobyshev > > > > How are you running qemu-iotests to make these tests fail? I tried to > > reproduce, but they just pass for me: > > > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > > [...] > > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > > Passed all 3 iotests > > > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > > 271 not run[11:21:20] [11:21:21] ... not > > suitable for this image format: raw > > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > > Not run: 271 > > Passed all 2 iotests > > > > Kevin > > > > As for the test 271, I imagine this might be caused by different request > alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If > I print the alignment out explicitly, I get: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request > alignment: 4096 Oh, do you have a 4k sector size? I think for me O_DIRECT only enforces 512 byte alignment. I can later try again on a 4k sector size loopback device or something. If so, I think that's worth mentioning in the commit message. > For the record, I'm running tests on ext4. I don't expect that this makes the difference, but xfs for me. > I'm not sure about the fuse tests though. Could it also have smth to do > with the underlying fs? Probably the same request alignment thing then. Kevin