Re: testing units in a specific order?
[Antoon Pardon] > Well maybe unit tests shouldn't care (Thats what I think you meant), Yup! > I care. Some methods are vital for the functionality of other methods. > So it the test for the first method fails it is very likely a number of > other methods will fail too. However I'm not interrested in the results > of those other tests in that case. Having to weed through all the test > results in order to check first if the vital methods are working before > checking other methods is cumbersome. > > Having the vital methods tested first and ignore the rest of the results > if they fail is much easier. So put the tests for the different kinds of methods into different test classes, and run the corresponding test suites in the order you want them to run. This is easy. Code like: test_classes = [FileStorageConnectionTests, FileStorageReconnectionTests, FileStorageInvqTests, FileStorageTimeoutTests, MappingStorageConnectionTests, MappingStorageTimeoutTests] def test_suite(): suite = unittest.TestSuite() for klass in test_classes: suite.addTest(unittest.makeSuite(klass)) return suite is common in large projects. unittest runs tests added to a suite in the order you add them (although _within_ a test class, the test methods are run in alphabetical order of method name -- when you want ordering, that's the wrong level to try to force it; forcing order is natural & easy at higher levels). -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
Op 2006-01-09, Tim Peters schreef <[EMAIL PROTECTED]>: > [Antoon Pardon] >> I have used unit tests now for a number of project. One thing >> that I dislike is it that the order in which the tests are done >> bears no relationship to the order they appear in the source. >> >> This makes using unit tests somewhat cumbersome. Is there some >> way to force the tests being done in a particular order? > > They're run in alphabetical order, sorting on the test methods' names. > For that reason some people name test methods like 'test_001', > 'test_002', ..., although unit tests really "shouldn't" case which > order they get run in. Well maybe unit tests shouldn't care (Thats what I think you meant), I care. Some methods are vital for the functionality of other methods. So it the test for the first method fails it is very likely a number of other methods will fail too. However I'm not interrested in the results of those other tests in that case. Having to weed through all the test results in order to check first if the vital methods are working before checking other methods is cumbersome. Having the vital methods tested first and ignore the rest of the results if they fail is much easier. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
Tim Peters <[EMAIL PROTECTED]> writes: > They're run in alphabetical order, sorting on the test methods' names. > For that reason some people name test methods like 'test_001', > 'test_002', ..., although unit tests really "shouldn't" case which > order they get run in. This seems sort of hard to do with "good" OO design. I.e. - I have some object foo with the requirement that "foo.setup" be run before "foo.process". I typically do unit tests to run foo.setup first, and the test for foo.process assumes that foo.setup has been run. The alternative would have the setup for "foo.process" run "foo.setup" as part of the prep. But that assumes that foo.setup worked properly. To make sure of that, I still need the unit test for foo.setup to run before the test for foo.process. I supposed the test would be better if the test of foo.process didn't expect foo.setup to be run first. But in that case, if foo.setup fails, running foo.process is only moderatly interesting - a failure may be a cascade from the failure of setup, and passing the test may not mean it'll pass if setup actually finishes properly. http://www.mired.org/home/mwm/ Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information. -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
Hi Michele! Michele Simionato wrote: > This looks new. Can you comment on how utestconvert.py work and how > reliable is it? It is rather old, but was never advertised much. PyPy used it to convert all its unit tests from unittests to py.test, when py.test was finished. The idea is quite simple, the script does a stupid rewrite of the typical unittest-syntax to py.test syntax: self.assertEqual(x, y) --> assert x == y self.assert_(some_expression) --> assert some_expression and so on. It works very well in "regular" unittest code that does not use any special unittests extensions. It might break for more complicated cases but (especially custom unittest extensions), as I said, it worked well with PyPy's unittests (which are quite many). In general, I think that py.test will at one point grow a collector for doctest/unittest test cases. We plan to do that since quite some time now (it is even part an issue in our tracker) but we currently don't have the resources to actually do so. Contributions are of course welcome ;-) Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
[Antoon Pardon] > I have used unit tests now for a number of project. One thing > that I dislike is it that the order in which the tests are done > bears no relationship to the order they appear in the source. > > This makes using unit tests somewhat cumbersome. Is there some > way to force the tests being done in a particular order? They're run in alphabetical order, sorting on the test methods' names. For that reason some people name test methods like 'test_001', 'test_002', ..., although unit tests really "shouldn't" case which order they get run in. Sometimes this is abused in a different way, by naming a setup kind of method starting with AAA and its corresponding teardown kind of method with zzz. You could presumably change the sort order by subclassing TestLoader and overriding its class-level .sortTestMethodsUsing attribute (which is `cmp` in TestLoader). Sounds painful and useless to me, though ;-) The source-code order isn't available in any case (unittest doesn't analyze source code). -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
Disclaimer: I am a contributor to the py-lib, of which py.test is part of (but have not worked on py.test until now). Michele Simionato wrote: > You could use py.test > Indeed. It has exactly this feature (together with many nifty others): all tests are run in the order they appear in the test-file. The website of the py-lib is at http://codespeak.net/py the documentation page for py.test is http://codespeak.net/py/current/doc/test.html If you like it so much (and I am being optimistic here :-) that you want to switch tests that use the stdlib unittest module over to py.test you can use a script called utestconvert.py which converts the unittest syntax over to py.test syntax. It can be found in the tool directory of the py-lib. Cheers, Carl Friedrich Bolz -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
Carl Friedrich Bolz wrote: > If you like it so much (and I am being optimistic here :-) that you want > to switch tests that use the stdlib unittest module over to py.test you > can use a script called utestconvert.py which converts the unittest > syntax over to py.test syntax. It can be found in the tool directory of > the py-lib. This looks new. Can you comment on how utestconvert.py work and how reliable is it? Thanks, Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: testing units in a specific order?
You could use py.test -- http://mail.python.org/mailman/listinfo/python-list
testing units in a specific order?
I have used unit tests now for a number of project. One thing that I dislike is it that the order in which the tests are done bears no relationship to the order they appear in the source. This makes using unit tests somewhat cumbersome. Is there some way to force the tests being done in a particular order? -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
