Hi folks, I've been working on a small project to:
- Extend the nasty depot to be more nasty and more configurable. - Enable the test suite to launch nasty depots. - Extend the test suite with the ability to run nasty-depot based stress tests. - File and fix bugs in the core code identified by this testing. I spoke to Brock at some length about whether or not the new test cases in cli/t_pkg_nasty.py should be in the 'cli' suite or perhaps in a new suite such as 'stress' which is not run by default. We're going to start with the tests living in 'cli' but they could move pretty easily if that doesn't work out well in practice. Let's not assume this is set in stone. The new tests run 10 iterations of each nasty test by default; there are two test classes. This stuff will run in parallel as usual with other tests. But if you are running serially, the new tests add ~180 seconds to the suite runtime. We can tune that impact down by dropping the default number of iterations if it's a big concern. I do want to make sure we at least run through this once on each test run in order to make sure it doesn't rot. The number of test iterations is configurable by environment variable, which should enable trivial integration with hudson. For example, you can do: NASTY_ITERS=500 python src/tests/run.py -j 8 -o t_pkg_nasty And you'll get 500 iterations of the testing. With the bugfixes applied, I have been able to run this stuff cleanly up to ~10000 iterations (I did this by adding more parallelism by hand) and it has been stable. There is much opportunity to improve this work-- we need to add testing (and associated bugfixes) for other areas not covered by this work. 7192182 pkgrecv traceback trying to setup request headers on a refetch 7192183 nasty depot testing should be part of core testing 7192447 transport.py has bad exception handling in do_search 7193408 variable name incorrect in error path when getting a p5i 7193416 client can traceback if manifest response is empty 7195048 contents -g can traceback if a temporary publisher has problems 7195051 catalog.attrs code needs improved armor against bogus responses 7195083 p5i.parse should discriminate "" versus None https://cr.opensolaris.org/action/browse/pkg/dp/nasty-testing/ -dp -- Daniel Price, Solaris Kernel Engineering _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
