Re: [Zope-dev] unit test policy questions
Hi! Marius Gedminas wrote: >> And if I use the -t option (-ttest_tool), most tests fail :( > > That is an indication that the tests aren't independent. I would > consider such a test suite to be broken. Well. In this case the tests are independent from each other, but not from the way tests are filtered: The -m filter is applied *before* test modules are imported. The -t filter is applied *after* importing all available test modules. Some nasty code in Zope 2 behaves differently if different modules are imported before running the tests. Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On Thu, Jul 29, 2010 at 02:52:36PM +0200, yuppie wrote: > Hi Martin! > > > Martin Aspeli wrote: > > On 29 July 2010 20:14, yuppie wrote: > >> Martin Aspeli wrote: > >>> I never do either. I install zc.recipe.testrunner in a buildout and > >>> use bin/test, which picks up tests in modules automatically. > >> > >> Sure. But do you always run all tests it picks up while working on a > >> specific test file? Or do you use bin/test with options that allow to > >> run specific files? > > > > See the -s and -t options. :) > > Well. It's not that simple. Today I tried to modify > Products.GenericSetup.tests.test_tool using CMF.buildout/trunk. > > I usually use the -m option, but that doesn't work in CMF.buildout > because it already uses that option. Ouch. Not a good idea, in my book. > So I tried to execute the module and noticed that's currently broken. > > The -s option doesn't allow to run a specific module. > > And if I use the -t option (-ttest_tool), most tests fail :( That is an indication that the tests aren't independent. I would consider such a test suite to be broken. Marius Gedminas -- http://pov.lt/ -- Zope 3/BlueBream consulting and development signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On Thu, Jul 29, 2010 at 4:35 PM, Tres Seaver wrote: > I don't believe that zope.testing's testrunner works without 'def > test_suite()'. Oh it does. Ever since 3.8.0 from mid 2009: - Testrunner automatically picks up descendants of unittest.TestCase in test modules, so you don't have to provide a test_suite() anymore. This just doesn't work for doctests and various forms of auto generating test cases and other fancier constructs. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 7/29/10 16:35 , Tres Seaver wrote: > yuppie wrote: >> Do you think it should become policy to remove "def test_suite():" where >> possible? I'm not sure if I like that because it is less explicit. > > I don't believe that zope.testing's testrunner works without 'def > test_suite()'. It seems to pick up my tests without it. Wichert. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 22:35, Tres Seaver wrote: > I don't believe that zope.testing's testrunner works without 'def > test_suite()'. Latter versions can detect unittest.TestCase-derived test suites automatically. For doctests you still need test_suite(). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 yuppie wrote: > Hi! > > > Hanno Schlichting wrote: >> On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli >> wrote: >>> On 29 July 2010 20:14, yuppie wrote: Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? >>> See the -s and -t options. :) >> Don't forget -m which runs exactly one module (file). >> >> I always use commands like: >> >> bin/test -s zope.package >> bin/test -s zope.package -m test_module >> bin/test -s zope.package -m test_module -t test_some_method > > If everybody is using this, nobody is using CMF.buildout for > development. Anyway. I fixed CMF.buildout: > http://svn.zope.org/?rev=115204&view=rev +1 for removing the 'if __name__ == "__main__"' boilerplate opportunistically. >> Especially since unittest and zope.testing can even discover >> TestSuites inside modules, it seems any kind of explicit configuration >> in the test files isn't needed anymore. > > Do you think it should become policy to remove "def test_suite():" where > possible? I'm not sure if I like that because it is less explicit. I don't believe that zope.testing's testrunner works without 'def test_suite()'. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkxRkb0ACgkQ+gerLs4ltQ4KugCgtZa1XK2SRcMIeu4e0BlZZOQ4 t2gAnjPZjjPewRWr0OnxOeezDn0crgb5 =8VTG -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On Thu, Jul 29, 2010 at 3:57 PM, yuppie wrote: > If everybody is using this, nobody is using CMF.buildout for > development. Anyway. I fixed CMF.buildout: > http://svn.zope.org/?rev=115204&view=rev I'm using CMF.buildout and was always annoyed by it overriding the -m option. But the CMF tests are fast enough, so I just ran the tests for a package at a time. > Do you think it should become policy to remove "def test_suite():" where > possible? I'm not sure if I like that because it is less explicit. I wouldn't make any decision at this point. Once we actually use unittest2 inside zope.testing we might want to reevaluate this. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/29/10 15:57 , yuppie wrote: > Hanno Schlichting wrote: >> On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli >> wrote: >>> On 29 July 2010 20:14, yuppie wrote: Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? >>> See the -s and -t options. :) >> Don't forget -m which runs exactly one module (file). >> >> I always use commands like: >> >> bin/test -s zope.package >> bin/test -s zope.package -m test_module >> bin/test -s zope.package -m test_module -t test_some_method > > If everybody is using this, nobody is using CMF.buildout for > development. Huh? I use it. I just don't use "-m" myself. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxRibYACgkQRAx5nvEhZLIE7wCfePls1EWIlJ5JlXmkk36kw1+y ZJIAoLpNA7ukrhN/bOV5ZfiCf+GhUbXY =h567 -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
Hi! Hanno Schlichting wrote: > On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli > wrote: >> On 29 July 2010 20:14, yuppie wrote: >>> Sure. But do you always run all tests it picks up while working on a >>> specific test file? Or do you use bin/test with options that allow to >>> run specific files? >> >> See the -s and -t options. :) > > Don't forget -m which runs exactly one module (file). > > I always use commands like: > > bin/test -s zope.package > bin/test -s zope.package -m test_module > bin/test -s zope.package -m test_module -t test_some_method If everybody is using this, nobody is using CMF.buildout for development. Anyway. I fixed CMF.buildout: http://svn.zope.org/?rev=115204&view=rev > Especially since unittest and zope.testing can even discover > TestSuites inside modules, it seems any kind of explicit configuration > in the test files isn't needed anymore. Do you think it should become policy to remove "def test_suite():" where possible? I'm not sure if I like that because it is less explicit. Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 07/29/2010 02:17 PM, Jens Vagelpohl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 7/29/10 14:00 , Martin Aspeli wrote: >> On 29 July 2010 19:26, yuppie wrote: >>> 1.) Is it still policy to add these lines? >>> >>> 2.) Is there a better solution for using zope testrunner than the one >>> shown above? >> >> I never do either. I install zc.recipe.testrunner in a buildout and >> use bin/test, which picks up tests in modules automatically. > > Same here. I have never in my life run tests by executing the module. > The options provided by the testrunner to pare down what I want to test > have always been sufficient for me. > > I have had a tendency to add these blurbs myself, thinking "I guess it's > useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, > it's not really useful anymore. Fullack. Even better: the current test runner even lets you avoid the test_suite function in many cases. -- Christian Theune · c...@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 07/29/2010 02:17 PM, Jens Vagelpohl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 7/29/10 14:00 , Martin Aspeli wrote: >> On 29 July 2010 19:26, yuppie wrote: >>> 1.) Is it still policy to add these lines? >>> >>> 2.) Is there a better solution for using zope testrunner than the one >>> shown above? >> >> I never do either. I install zc.recipe.testrunner in a buildout and >> use bin/test, which picks up tests in modules automatically. > > Same here. I have never in my life run tests by executing the module. > The options provided by the testrunner to pare down what I want to test > have always been sufficient for me. > > I have had a tendency to add these blurbs myself, thinking "I guess it's > useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, > it's not really useful anymore. Fullack. Even better: the current test runner even lets you avoid the test_suite function in many cases. -- Christian Theune · c...@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
Hi Martin! Martin Aspeli wrote: > On 29 July 2010 20:14, yuppie wrote: >> Martin Aspeli wrote: >>> I never do either. I install zc.recipe.testrunner in a buildout and >>> use bin/test, which picks up tests in modules automatically. >> >> Sure. But do you always run all tests it picks up while working on a >> specific test file? Or do you use bin/test with options that allow to >> run specific files? > > See the -s and -t options. :) Well. It's not that simple. Today I tried to modify Products.GenericSetup.tests.test_tool using CMF.buildout/trunk. I usually use the -m option, but that doesn't work in CMF.buildout because it already uses that option. So I tried to execute the module and noticed that's currently broken. The -s option doesn't allow to run a specific module. And if I use the -t option (-ttest_tool), most tests fail :( Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On Thu, Jul 29, 2010 at 2:23 PM, Martin Aspeli wrote: > On 29 July 2010 20:14, yuppie wrote: >> Sure. But do you always run all tests it picks up while working on a >> specific test file? Or do you use bin/test with options that allow to >> run specific files? > > See the -s and -t options. :) Don't forget -m which runs exactly one module (file). I always use commands like: bin/test -s zope.package bin/test -s zope.package -m test_module bin/test -s zope.package -m test_module -t test_some_method Especially since unittest and zope.testing can even discover TestSuites inside modules, it seems any kind of explicit configuration in the test files isn't needed anymore. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 20:14, yuppie wrote: > Martin Aspeli wrote: >> I never do either. I install zc.recipe.testrunner in a buildout and >> use bin/test, which picks up tests in modules automatically. > > Sure. But do you always run all tests it picks up while working on a > specific test file? Or do you use bin/test with options that allow to > run specific files? See the -s and -t options. :) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
Am 29.07.2010, 14:14 Uhr, schrieb yuppie : > Sure. But do you always run all tests it picks up while working on a > specific test file? Or do you use bin/test with options that allow to > run specific files? I try and limit my tests to the package or module I'm working on. I've been bitten by my occasional use and the changes in some of the switches but -s gives you most of the control you need. Charlie -- Charlie Clark Managing Director Clark Consulting & Research German Office Helmholtzstr. 20 Düsseldorf D- 40215 Tel: +49-211-600-3657 Mobile: +49-178-782-6226 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/29/10 14:00 , Martin Aspeli wrote: > On 29 July 2010 19:26, yuppie wrote: >> 1.) Is it still policy to add these lines? >> >> 2.) Is there a better solution for using zope testrunner than the one >> shown above? > > I never do either. I install zc.recipe.testrunner in a buildout and > use bin/test, which picks up tests in modules automatically. Same here. I have never in my life run tests by executing the module. The options provided by the testrunner to pare down what I want to test have always been sufficient for me. I have had a tendency to add these blurbs myself, thinking "I guess it's useful for *someone*". But IMHO it's like those CVS/SVN $Id$ snippets, it's not really useful anymore. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkxRcUYACgkQRAx5nvEhZLLubgCgpCyXpHVToolxYzsfGGubmdHi n+YAnjCVZvY0WhoWZ09015unDWvnMIAx =O8yt -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
Martin Aspeli wrote: > I never do either. I install zc.recipe.testrunner in a buildout and > use bin/test, which picks up tests in modules automatically. Sure. But do you always run all tests it picks up while working on a specific test file? Or do you use bin/test with options that allow to run specific files? Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] unit test policy questions
On 29 July 2010 19:26, yuppie wrote: > Hi! > > > Traditionally the last two lines of unit test files look like this: > > if __name__ == '__main__': > unittest.main(defaultTest='test_suite') > > That makes it easy to run the tests of a specific file. But it doesn't > work with tests that require the zope testrunner. AFAICS something like > this is needed instead: > > if __name__ == '__main__': > from zope.testing.testrunner import run > run(['-m', 'test_foo', '--test-path', '.']) > > > Questions: > -- > > 1.) Is it still policy to add these lines? > > 2.) Is there a better solution for using zope testrunner than the one > shown above? I never do either. I install zc.recipe.testrunner in a buildout and use bin/test, which picks up tests in modules automatically. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] unit test policy questions
Hi! Traditionally the last two lines of unit test files look like this: if __name__ == '__main__': unittest.main(defaultTest='test_suite') That makes it easy to run the tests of a specific file. But it doesn't work with tests that require the zope testrunner. AFAICS something like this is needed instead: if __name__ == '__main__': from zope.testing.testrunner import run run(['-m', 'test_foo', '--test-path', '.']) Questions: -- 1.) Is it still policy to add these lines? 2.) Is there a better solution for using zope testrunner than the one shown above? Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
