Quoting Dustin Kirkland ([email protected]): > On Thu, Aug 4, 2011 at 12:32 PM, Serge E. Hallyn > <[email protected]> wrote: > > It was suggested (URL) that we should look at the phoronix testsuite for > > testing ecryptfs. Phoronix very nicely automates the installation and > > running > > of many benchmarks. It doesn't however appear to have any actual > > correctness > > tests. It may be worth it to do periodic tests comparing some tests like > > fio > > and dbench (both of which phoronix supports) with and without ecryptfs. > > However, these sorts of tests would be harder to use since we can't > > meaningfully run them on VMs. > > > > The posix test suite, AIUI, tests other kernel features like shm, but > > does not test proper posix file behavior.
Ah, but I just saw http://lwn.net/Articles/276617/. That's promising. > > The LTP testsuite is one we sould look at. The fs testsuite has a set of > > both fs stress tests (like growfiles) and correctness tests (like inode01). > > > > We also could write some testcases of our own. I've started a list of > > potential tests at http://wiki.ubuntu.com/ecryptfs-tests. Please feel > > free to add entries. Perhaps at the next UDS we can have a session > > going over the accumulated list, prioritizing, and setting a definite > > implementation plan. ("i.e. serge, go code it, based on ltp" :) > > > > My own recommendation would be that we start by writing a wrapper which > > fetches ltp and runs the fs testsuite on ext4 and ecryptfs-on-ext4. I'll > > happily script that and package the script as ecryptfs-testsuite. Then > > we can move on to writing some of our own testcases. > > > > How does that sound to people? If you feel phoronix testsuite should > > also be done, then I'd like to hear from someone who can volunteer some > > bare metal, long term, for repeated tests. > > +1 for automated testing, of course. Phoronix is good for long term > benchmarking, noticing major regressions, etc. But > functional/correctness testing is more important to me as a user and > one of the maintainers. Agreed. > See the end of the ecryptfs-setup-private script for a very basic set > of read/write correctness tests. We should extend that concept, Yeah, cool, that's a starting point. Thanks. > reading and writing tens of thousands of files, large and small, > hundreds of megabytes, and also dig through dmesg and the logs looking > for ecryptfs errors, as those are starting to creep up more and more. Hm, that starts to worry me. The reason I wanted to compile a list of useful test cases and discuss them before digging in, is that I've seen it happen too often where a bunch of crappy tests are written which are testing side effects of current implementation rather than important indicators of correctness, and eventually the test suite becomes more a nuisance than a help. Looking for kernel msgs worries me in that same way. thanks, -serge _______________________________________________ Mailing list: https://launchpad.net/~ecryptfs-devel Post to : [email protected] Unsubscribe : https://launchpad.net/~ecryptfs-devel More help : https://help.launchpad.net/ListHelp
