-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 03/28/2011 10:04 AM, Jim Fulton wrote: > On Sun, Mar 27, 2011 at 11:13 AM, Martin Aspeli > <optilude+li...@gmail.com> wrote: >> Hi, >> >> On 27 March 2011 15:54, Uli Fouquet <u...@gnufix.de> wrote: >> >>> The (limited) experiences with py.test, however, were awesome. Some >>> points that are quite cool IMHO: >>> >>> - Easy finding of tests: just write some ``test_function`` in a >>> ``test_module`` and it will be found and executed. That also makes >>> py.test tests more readable and maybe more intuitive. >> >> I'm not sure this is always a good idea. It feels a bit implicit, and >> having a base class isn't really a big problem, IMHO. It seems a bit >> like the kind of thing that sounds cool (look, it's even easier!), but >> in practice makes little difference. > > +1 > +1 > +1 > > This is especially important for doctests (and manuel) or any > situation where setup is important and where you can't really guess. > >>> - py.test is more widespread in the Python community (that's my >>> impression; I can't proof it) >> >> What about nose? > > It looks to me like a layerish mechanism might be possible in nose, or > at least like > zope.testing layers could be integrated with nose. > >> >>> - Support of unittest/unittest2: you can write standard lib setups >>> (defining TestCases; no need to also write testsuite-setup stuff) and >>> they will be found/executed. zope.testrunner for instance does not >>> support the new `setUpClass`/`tearDownClass` concept of unittest2 >>> (yes, you would use layers in that case; but it might be nice if >>> zope.testrunner would support also class-wide fixtures in >>> unittest2-style; people from other worlds might expect that to work). >> >> zope.testing should definitely gain support for the new unittest2 >> hooks. That wouldn't be very hard, though. ;-) > > I assume you mean zope.testrunner. > >> >>> Main drawbacks I see on py.test side are: >>> >>> - Lack of layer support (yet). Maybe we can do something about that in >>> `zope.pytest` based on `plone.testing.layer`. >>> >>> - Limited doctest support. It is quite difficult (AFAIK) to define >>> fixtures for doctests or to even set the usual doctest options >>> (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests >>> are simply collected and executed and not much finetuning is possible. >> >> With zope.testrunner, you *do* need a test_suite method to run >> doctests. I think that's a good thing. Look at plone.testing's README >> for examples. > > Again, +1 > > If I were to use nose or py.tests, I would want to adopt an explicit > style, which I believe is possible w nose.
FWIW, I thought the same, but haven't missed 'test_suite' appreciably after a week or so of acclimatization to the nose discovery mechanism. The only wrinkle I know of is that one doesn't use base classes for test cases where the base class itself derives from unittest.TestCase: instead, the base is just used as a "pure mixin". >> FWIW, I think we should stop using .txt doctests for unit tests. > > I disagree, of course. > >> Doctests should be used to test *documentation* ("the examples are >> valid"). > > Manuel is *much* better for that. (Of course, manuel is arguably a form of > doctest.) The Sphinx doctest integration[1] ('make doctest') supports both classic "interpreter prompt" doctests and a more useful (IMHO) "code-output" form, such as: .. testcode:: 1+1 # this will give no output! print 2+2 # this will give output .. testoutput:: 4 [1] http://sphinx.pocoo.org/ext/doctest.html >> For actual unit tests, writing tests in a unittest class is >> almost always better in the long run. doctests don't scale well and >> discourage the kind of ad-hoc "this seems broken, I'll just write a >> quick test" or "I just fixed a bug, better add a regression test" >> testing. > > You're just not using them correctly. :) The vast majority of the doctest testcases in zope.* packages fall into this category: poor isolation, lots of edge cases which would obscure any real narrative docs, of which there are almost none. I believe the conflict is intrinsic, here, and not just an accident of careless / naive implementation: exercising all the preconditions of the contract of the function-under-test makes for really poor documentation, but is essential to good testing. > One thing I hate about unittest is the javiotic ceremony it involves. > Doctests can cut down on the clutter a lot. I believe that that is > py.test's strength as well.. "I'll take explicit for $1000, Alex." > More generally, I'd love to see us adopt another test runner so that > we can stop maintianing zope.testrunner. When it was written at > the turn of the century, there weren't good alternatives. Personally, > I think maintaining it is boring. Nose and coverage have been pretty good to repoze.* and pyramid_*. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2QnwcACgkQ+gerLs4ltQ74FQCdGX9twswg6eAy9cnpABAZRrbo pvoAniGfpBliwnFjqFrV5uqpFcePI2tk =cqN7 -----END PGP SIGNATURE----- _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )