Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 12/11/23 15:32, Andrey Drobyshev wrote:
> There're tests whose logic implies running without O_DIRECT set,
> otherwise they fail when running iotests in '-nocache' mode. For these
> tests let's add _require_no_o_direct() helper which can be put in the
> preabmle and which makes sure '-nocache' isn't set. Use it to skip
> running the following tests:
>
> * 271: creates files with unaligned sizes, thus producing multiple
> errors like:
>
> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write'
> permission without 'resize': Image size is not a multiple of request alignment
>
> * 308, file-io-error: use fuse exports. Though fuse does have
> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html)
> we aren't using it yet, thus getting errors like:
>
> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open
> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT
>
> Signed-off-by: Andrey Drobyshev
> ---
> tests/qemu-iotests/271 | 1 +
> tests/qemu-iotests/308 | 2 ++
> tests/qemu-iotests/common.rc | 7 +++
> tests/qemu-iotests/tests/file-io-error | 1 +
> 4 files changed, 11 insertions(+)
>
> diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271
> index 59a6fafa2f..1424b6954d 100755
> --- a/tests/qemu-iotests/271
> +++ b/tests/qemu-iotests/271
> @@ -44,6 +44,7 @@ _supported_fmt qcow2
> _supported_proto file nfs
> _supported_os Linux
> _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file
> refcount_bits=1[^0-9]
> +_require_no_o_direct
>
> l2_offset=$((0x4))
>
> diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308
> index de12b2b1b9..535455e5b1 100755
> --- a/tests/qemu-iotests/308
> +++ b/tests/qemu-iotests/308
> @@ -52,6 +52,8 @@ _unsupported_fmt vpc
> _supported_proto file # We create the FUSE export manually
> _supported_os Linux # We need /dev/urandom
>
> +_require_no_o_direct
> +
> # $1: Export ID
> # $2: Options (beyond the node-name and ID)
> # $3: Expected return value (defaults to 'return')
> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
> index 95c12577dd..f61eae73b4 100644
> --- a/tests/qemu-iotests/common.rc
> +++ b/tests/qemu-iotests/common.rc
> @@ -857,6 +857,13 @@ _check_o_direct()
> [[ "$out" != *"O_DIRECT"* ]]
> }
>
> +_require_no_o_direct()
> +{
> +if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then
> +_notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)"
> +fi
> +}
> +
> _require_o_direct()
> {
> if ! _check_o_direct; then
> diff --git a/tests/qemu-iotests/tests/file-io-error
> b/tests/qemu-iotests/tests/file-io-error
> index 88ee5f670c..2b8dc7f009 100755
> --- a/tests/qemu-iotests/tests/file-io-error
> +++ b/tests/qemu-iotests/tests/file-io-error
> @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
> # Format-agnostic (we do not use any), but we do test the file protocol
> _supported_proto file
> _require_drivers blkdebug null-co
> +_require_no_o_direct
>
> if [ "$IMGOPTSSYNTAX" = "true" ]; then
> # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts,
Ping
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 1/11/24 14:53, Andrey Drobyshev wrote:
> On 12/11/23 15:32, Andrey Drobyshev wrote:
>> There're tests whose logic implies running without O_DIRECT set,
>> otherwise they fail when running iotests in '-nocache' mode. For these
>> tests let's add _require_no_o_direct() helper which can be put in the
>> preabmle and which makes sure '-nocache' isn't set. Use it to skip
>> running the following tests:
>>
>> * 271: creates files with unaligned sizes, thus producing multiple
>> errors like:
>>
>> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write'
>> permission without 'resize': Image size is not a multiple of request
>> alignment
>>
>> * 308, file-io-error: use fuse exports. Though fuse does have
>> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html)
>> we aren't using it yet, thus getting errors like:
>>
>> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open
>> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT
>>
>> Signed-off-by: Andrey Drobyshev
>> ---
>> tests/qemu-iotests/271 | 1 +
>> tests/qemu-iotests/308 | 2 ++
>> tests/qemu-iotests/common.rc | 7 +++
>> tests/qemu-iotests/tests/file-io-error | 1 +
>> 4 files changed, 11 insertions(+)
>>
>> diff --git a/tests/qemu-iotests/271 b/tests/qemu-iotests/271
>> index 59a6fafa2f..1424b6954d 100755
>> --- a/tests/qemu-iotests/271
>> +++ b/tests/qemu-iotests/271
>> @@ -44,6 +44,7 @@ _supported_fmt qcow2
>> _supported_proto file nfs
>> _supported_os Linux
>> _unsupported_imgopts extended_l2 compat=0.10 cluster_size data_file
>> refcount_bits=1[^0-9]
>> +_require_no_o_direct
>>
>> l2_offset=$((0x4))
>>
>> diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308
>> index de12b2b1b9..535455e5b1 100755
>> --- a/tests/qemu-iotests/308
>> +++ b/tests/qemu-iotests/308
>> @@ -52,6 +52,8 @@ _unsupported_fmt vpc
>> _supported_proto file # We create the FUSE export manually
>> _supported_os Linux # We need /dev/urandom
>>
>> +_require_no_o_direct
>> +
>> # $1: Export ID
>> # $2: Options (beyond the node-name and ID)
>> # $3: Expected return value (defaults to 'return')
>> diff --git a/tests/qemu-iotests/common.rc b/tests/qemu-iotests/common.rc
>> index 95c12577dd..f61eae73b4 100644
>> --- a/tests/qemu-iotests/common.rc
>> +++ b/tests/qemu-iotests/common.rc
>> @@ -857,6 +857,13 @@ _check_o_direct()
>> [[ "$out" != *"O_DIRECT"* ]]
>> }
>>
>> +_require_no_o_direct()
>> +{
>> +if [ $CACHEMODE == "none" ] || [ $CACHEMODE == "directsync" ]; then
>> +_notrun "not suitable for cache mode: $CACHEMODE (implies O_DIRECT)"
>> +fi
>> +}
>> +
>> _require_o_direct()
>> {
>> if ! _check_o_direct; then
>> diff --git a/tests/qemu-iotests/tests/file-io-error
>> b/tests/qemu-iotests/tests/file-io-error
>> index 88ee5f670c..2b8dc7f009 100755
>> --- a/tests/qemu-iotests/tests/file-io-error
>> +++ b/tests/qemu-iotests/tests/file-io-error
>> @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
>> # Format-agnostic (we do not use any), but we do test the file protocol
>> _supported_proto file
>> _require_drivers blkdebug null-co
>> +_require_no_o_direct
>>
>> if [ "$IMGOPTSSYNTAX" = "true" ]; then
>> # We need `$QEMU_IO -f file` to work; IMGOPTSSYNTAX uses --image-opts,
>
> Ping
Ping
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On Mon, Dec 11, 2023 at 03:32:23PM +0200, Andrey Drobyshev wrote: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip preamble > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment I wonder if we can instead tweak the test to use larger sizes such that all accesses ARE aligned even with O_DIRECT. /me goes and reads the test... # Note that the image size is not a multiple of the cluster size _reset_img 2083k Ah - we really DO have a test that depends on odd sizing; where changing it to be more nicely aligned will break other assumptions in the test. > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT And I agree that this one is beyond our control, so adding skip support makes sense. > > Signed-off-by: Andrey Drobyshev > --- > tests/qemu-iotests/271 | 1 + > tests/qemu-iotests/308 | 2 ++ > tests/qemu-iotests/common.rc | 7 +++ > tests/qemu-iotests/tests/file-io-error | 1 + > 4 files changed, 11 insertions(+) Reviewed-by: Eric Blake -- Eric Blake, Principal Software Engineer Red Hat, Inc. Virtualization: qemu.org | libguestfs.org
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > There're tests whose logic implies running without O_DIRECT set, > otherwise they fail when running iotests in '-nocache' mode. For these > tests let's add _require_no_o_direct() helper which can be put in the > preabmle and which makes sure '-nocache' isn't set. Use it to skip > running the following tests: > > * 271: creates files with unaligned sizes, thus producing multiple > errors like: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request alignment > > * 308, file-io-error: use fuse exports. Though fuse does have > 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > we aren't using it yet, thus getting errors like: > > qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > > Signed-off-by: Andrey Drobyshev How are you running qemu-iotests to make these tests fail? I tried to reproduce, but they just pass for me: $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error [...] 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) Passed all 3 iotests $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error 271 not run[11:21:20] [11:21:21] ... not suitable for this image format: raw 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) Not run: 271 Passed all 2 iotests Kevin
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
On 1/26/24 12:24, Kevin Wolf wrote: > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: >> There're tests whose logic implies running without O_DIRECT set, >> otherwise they fail when running iotests in '-nocache' mode. For these >> tests let's add _require_no_o_direct() helper which can be put in the >> preabmle and which makes sure '-nocache' isn't set. Use it to skip >> running the following tests: >> >> * 271: creates files with unaligned sizes, thus producing multiple >> errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' >> permission without 'resize': Image size is not a multiple of request >> alignment >> >> * 308, file-io-error: use fuse exports. Though fuse does have >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) >> we aren't using it yet, thus getting errors like: >> >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT >> >> Signed-off-by: Andrey Drobyshev > > How are you running qemu-iotests to make these tests fail? I tried to > reproduce, but they just pass for me: > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > [...] > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > Passed all 3 iotests > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > 271 not run[11:21:20] [11:21:21] ... not > suitable for this image format: raw > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > Not run: 271 > Passed all 2 iotests > > Kevin > As for the test 271, I imagine this might be caused by different request alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If I print the alignment out explicitly, I get: qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' permission without 'resize': Image size is not a multiple of request alignment: 4096 For the record, I'm running tests on ext4. I'm not sure about the fuse tests though. Could it also have smth to do with the underlying fs? Andrey
Re: [PATCH] iotests: don't run tests requiring cached writes in '-nocache' mode
Am 26.01.2024 um 12:04 hat Andrey Drobyshev geschrieben: > On 1/26/24 12:24, Kevin Wolf wrote: > > Am 11.12.2023 um 14:32 hat Andrey Drobyshev geschrieben: > >> There're tests whose logic implies running without O_DIRECT set, > >> otherwise they fail when running iotests in '-nocache' mode. For these > >> tests let's add _require_no_o_direct() helper which can be put in the > >> preabmle and which makes sure '-nocache' isn't set. Use it to skip > >> running the following tests: > >> > >> * 271: creates files with unaligned sizes, thus producing multiple > >> errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > >> permission without 'resize': Image size is not a multiple of request > >> alignment > >> > >> * 308, file-io-error: use fuse exports. Though fuse does have > >> 'direct-io' mode (see https://docs.kernel.org/filesystems/fuse-io.html) > >> we aren't using it yet, thus getting errors like: > >> > >> qemu-io: can't open device /path/to/t.qcow2.fuse: Could not open > >> '/path/to/t.qcow2.fuse': filesystem does not support O_DIRECT > >> > >> Signed-off-by: Andrey Drobyshev > > > > How are you running qemu-iotests to make these tests fail? I tried to > > reproduce, but they just pass for me: > > > > $ tests/qemu-iotests/check -qcow2 -nocache 271 308 file-io-error > > [...] > > 271 pass [11:20:50] [11:21:11] 21.1s (last: 20.4s) > > 308 pass [11:21:11] [11:21:14] 3.3s (last: 3.3s) > > file-io-error pass [11:21:14] [11:21:14] 0.3s (last: 0.3s) > > Passed all 3 iotests > > > > $ tests/qemu-iotests/check -raw -nocache 271 308 file-io-error > > 271 not run[11:21:20] [11:21:21] ... not > > suitable for this image format: raw > > 308 pass [11:21:21] [11:21:24] 3.8s (last: 2.8s) > > file-io-error pass [11:21:24] [11:21:25] 0.3s (last: 0.3s) > > Not run: 271 > > Passed all 2 iotests > > > > Kevin > > > > As for the test 271, I imagine this might be caused by different request > alignment. The failure occurs in block.c, bdrv_node_refresh_perm(). If > I print the alignment out explicitly, I get: > > qemu-io: can't open device /path/to/t.qcow2.raw: Cannot get 'write' > permission without 'resize': Image size is not a multiple of request > alignment: 4096 Oh, do you have a 4k sector size? I think for me O_DIRECT only enforces 512 byte alignment. I can later try again on a 4k sector size loopback device or something. If so, I think that's worth mentioning in the commit message. > For the record, I'm running tests on ext4. I don't expect that this makes the difference, but xfs for me. > I'm not sure about the fuse tests though. Could it also have smth to do > with the underlying fs? Probably the same request alignment thing then. Kevin
