Re: [PATCH 0/3] Improve block device testing coverage

[Darrick J. Wong]

On Fri, Mar 31, 2017 at 03:11:28PM +, Bart Van Assche wrote:
> On Fri, 2017-03-31 at 13:02 +0300, Dmitry Monakhov wrote:
> > Another good example may be a bug with dirty page cache after blkdiscard
> > https://lkml.org/lkml/2017/3/22/789 . This simple bug  result in crappy
> > fsimage if mkfs relay on discard_zeroes_data behaviour.
> > So IMHO basic blkdev test coverage is important filesystem testing. i.e.
> > important for xfstests.
> 
> Mixing up filesystem tests and block layer / block driver tests in the same
> directory is completely wrong. Block driver developers will be primarily
> interested in the block tests and may want to skip the filesystem tests.
> Filesystem developers will probably run the block tests only once and will
> likely run the filesystem tests repeatedly. Mixing up different kinds of
> tests in the same directory makes it unnecessarily hard to run block and
> filesystem tests separately.

During LSF I had started to wonder if we should just create a new
FSTYP=blockdev fs type with a no-op mkfs & mount.  "_require_fs generic"
could be taught to ignore FSTYP=blockdev; blockdev tests that should
work on all block devices can stay in tests/generic, and blockdev tests
that require specific features or complicated setup can go in
tests/blockdev.

The benefit (for the fs developers, anyway) of having complex block
device setup code helper functions in common/ is that then we can also
start writing tests to see how the fs reacts with more complex storage
setups.  We already have some of that for dm_{thin,flakey,delay,error}.

That way we keep the tests together and make it easy to run them (when
applicable) as part of regular fs testing, and avoid the situation where
bdevtests and xfstests slowly drift apart in terms of behaviors and
command line switches.

The downside ofc is the potential for bloat. :)

(The blockdev fallocate tests fit the fs/block split awkwardly --
they call what is nominally a fs feature on something that isn't itself
a filesystem...)

 Just my 5c.

--D

> 
> Bart.--
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 0/3] Improve block device testing coverage

[Bart Van Assche]

On Fri, 2017-03-31 at 13:02 +0300, Dmitry Monakhov wrote:
> Another good example may be a bug with dirty page cache after blkdiscard
> https://lkml.org/lkml/2017/3/22/789 . This simple bug  result in crappy
> fsimage if mkfs relay on discard_zeroes_data behaviour.
> So IMHO basic blkdev test coverage is important filesystem testing. i.e.
> important for xfstests.

Mixing up filesystem tests and block layer / block driver tests in the same
directory is completely wrong. Block driver developers will be primarily
interested in the block tests and may want to skip the filesystem tests.
Filesystem developers will probably run the block tests only once and will
likely run the filesystem tests repeatedly. Mixing up different kinds of
tests in the same directory makes it unnecessarily hard to run block and
filesystem tests separately.

Bart.

Re: [PATCH 0/3] Improve block device testing coverage

[Dmitry Monakhov]

Eryu Guan  writes:

> On Fri, Mar 31, 2017 at 10:43:19AM +0300, Dmitry Monakhov wrote:
>> Christoph Hellwig  writes:
>> 
>> > On Thu, Mar 30, 2017 at 05:19:01PM +0400, Dmitry Monakhov wrote:
>> >> During LSFMM we have discussed how to test lower-backend of linux 
>> >> IO-stack.
>> >> Common opinion was that xfstests is the most obvious solution which cover
>> >> most of use cases filesystem care about.
>> >> 
>> >> I'm working on integration T10-DIF/DIF data integrity features to ext4,
>> >> for that reason we need to be shure that linux integrity framework is
>> >> in working state, which is currently broken in several places.
>> >> 
>> >> In fact, it is relatively simple to add basic coverage tests for basic
>> >> IO operations over virtual device with integrity support. All we need
>> >> is to add lio target support.
>> >
>> > First:  Thanks for adding block layer testing!
>> >
>> > Second: even more so than Darrick's blockdev fallocate test this is
>> > the wrong place.  If I run xfstests I want to test my file system,
>> > not random block device features.  Please start a proper block device
>> > testsuite instead, possibly by copy and pasting code from xfstests.
>> Fair enough. I also not happy to place blkdev feature  to tests/generic
>> namespace. But altearnative to fork xfstests infrastructure to dedicated
>> test-framework only for blkdevice seems not very good. Because fork is
>> always pain. I already maintain one internal fork of xfstests which
>> tests our Vituozzo's speciffic features.
>> 
>> May be it would be reasonable idea to add didicated namespace
>> 'tests/blockdev' in xfstests, and move all blkdev related tests here?
>> IMHO this is good idea. Because filesystem relay on some basic
>> features from blkdev which should be tested explicitly, because
>> implicit testing is too hard to debug/investigation.
>
> I'm not sure if xfstests is the right place for blockdev tests,
> especially for the pure blockdev level features (at least Darrick's
> blockdev tests are testing fallocate(2) interface).
Another good example may be a bug with dirty page cache after blkdiscard
https://lkml.org/lkml/2017/3/22/789 . This simple bug  result in crappy
fsimage if mkfs relay on discard_zeroes_data behaviour.
So IMHO basic blkdev test coverage is important filesystem testing. i.e.
important for xfstests.
>
> But yeah, a new tests/blockdev dir would be good if we eventually decide
> adding blockdev tests to xfstests, so they're not run by default when
> people want to test filesystems.
>
> Thanks,
> Eryu

Re: [PATCH 0/3] Improve block device testing coverage

[Eryu Guan]

On Fri, Mar 31, 2017 at 10:43:19AM +0300, Dmitry Monakhov wrote:
> Christoph Hellwig  writes:
> 
> > On Thu, Mar 30, 2017 at 05:19:01PM +0400, Dmitry Monakhov wrote:
> >> During LSFMM we have discussed how to test lower-backend of linux IO-stack.
> >> Common opinion was that xfstests is the most obvious solution which cover
> >> most of use cases filesystem care about.
> >> 
> >> I'm working on integration T10-DIF/DIF data integrity features to ext4,
> >> for that reason we need to be shure that linux integrity framework is
> >> in working state, which is currently broken in several places.
> >> 
> >> In fact, it is relatively simple to add basic coverage tests for basic
> >> IO operations over virtual device with integrity support. All we need
> >> is to add lio target support.
> >
> > First:  Thanks for adding block layer testing!
> >
> > Second: even more so than Darrick's blockdev fallocate test this is
> > the wrong place.  If I run xfstests I want to test my file system,
> > not random block device features.  Please start a proper block device
> > testsuite instead, possibly by copy and pasting code from xfstests.
> Fair enough. I also not happy to place blkdev feature  to tests/generic
> namespace. But altearnative to fork xfstests infrastructure to dedicated
> test-framework only for blkdevice seems not very good. Because fork is
> always pain. I already maintain one internal fork of xfstests which
> tests our Vituozzo's speciffic features.
> 
> May be it would be reasonable idea to add didicated namespace
> 'tests/blockdev' in xfstests, and move all blkdev related tests here?
> IMHO this is good idea. Because filesystem relay on some basic
> features from blkdev which should be tested explicitly, because
> implicit testing is too hard to debug/investigation.

I'm not sure if xfstests is the right place for blockdev tests,
especially for the pure blockdev level features (at least Darrick's
blockdev tests are testing fallocate(2) interface).

But yeah, a new tests/blockdev dir would be good if we eventually decide
adding blockdev tests to xfstests, so they're not run by default when
people want to test filesystems.

Thanks,
Eryu

Re: [PATCH 0/3] Improve block device testing coverage

[Dmitry Monakhov]

Christoph Hellwig  writes:

> On Thu, Mar 30, 2017 at 05:19:01PM +0400, Dmitry Monakhov wrote:
>> During LSFMM we have discussed how to test lower-backend of linux IO-stack.
>> Common opinion was that xfstests is the most obvious solution which cover
>> most of use cases filesystem care about.
>> 
>> I'm working on integration T10-DIF/DIF data integrity features to ext4,
>> for that reason we need to be shure that linux integrity framework is
>> in working state, which is currently broken in several places.
>> 
>> In fact, it is relatively simple to add basic coverage tests for basic
>> IO operations over virtual device with integrity support. All we need
>> is to add lio target support.
>
> First:  Thanks for adding block layer testing!
>
> Second: even more so than Darrick's blockdev fallocate test this is
> the wrong place.  If I run xfstests I want to test my file system,
> not random block device features.  Please start a proper block device
> testsuite instead, possibly by copy and pasting code from xfstests.
Fair enough. I also not happy to place blkdev feature  to tests/generic
namespace. But altearnative to fork xfstests infrastructure to dedicated
test-framework only for blkdevice seems not very good. Because fork is
always pain. I already maintain one internal fork of xfstests which
tests our Vituozzo's speciffic features.

May be it would be reasonable idea to add didicated namespace
'tests/blockdev' in xfstests, and move all blkdev related tests here?
IMHO this is good idea. Because filesystem relay on some basic
features from blkdev which should be tested explicitly, because
implicit testing is too hard to debug/investigation.

>
> That's how I started the test suite for qemu's block layer for example.
Do you mean qemu/tests/qemu-iotests ?

Re: [PATCH 0/3] Improve block device testing coverage

[Christoph Hellwig]

On Thu, Mar 30, 2017 at 05:19:01PM +0400, Dmitry Monakhov wrote:
> During LSFMM we have discussed how to test lower-backend of linux IO-stack.
> Common opinion was that xfstests is the most obvious solution which cover
> most of use cases filesystem care about.
> 
> I'm working on integration T10-DIF/DIF data integrity features to ext4,
> for that reason we need to be shure that linux integrity framework is
> in working state, which is currently broken in several places.
> 
> In fact, it is relatively simple to add basic coverage tests for basic
> IO operations over virtual device with integrity support. All we need
> is to add lio target support.

First:  Thanks for adding block layer testing!

Second: even more so than Darrick's blockdev fallocate test this is
the wrong place.  If I run xfstests I want to test my file system,
not random block device features.  Please start a proper block device
testsuite instead, possibly by copy and pasting code from xfstests.

That's how I started the test suite for qemu's block layer for example.

[PATCH 0/3] Improve block device testing coverage

[Dmitry Monakhov]

During LSFMM we have discussed how to test lower-backend of linux IO-stack.
Common opinion was that xfstests is the most obvious solution which cover
most of use cases filesystem care about.

I'm working on integration T10-DIF/DIF data integrity features to ext4,
for that reason we need to be shure that linux integrity framework is
in working state, which is currently broken in several places.

In fact, it is relatively simple to add basic coverage tests for basic
IO operations over virtual device with integrity support. All we need
is to add lio target support.

TOC:
add lio target helpers
add test: generic/420 check information lead for lio-fileio
add test: generic/421 basic blockdev T10-DIF-TYPE1 IO