On Sun, Aug 2, 2009 at 6:34 PM, Jim Fulton<j...@zope.com> wrote: > 2. Some of the tests only pass if run separately, due to test > interactions. Presumably, this means that other tests aren't cleaning > up after themselves. I think we need a standard automated way to run > each package's tests separately. I think some folks are working on > that.
I had a similar problem with the Plone trunk tests and my solution to that is http://pypi.python.org/pypi/plone.recipe.alltests (licensed as ZPL). My example usage of it is at http://svn.plone.org/svn/plone/buildouts/plone-coredev/trunk/tests.cfg The main idea here is to generate a "./bin/alltests" script, which internally runs the tests on a package by package basis (using a simple os.system call to the zc.recipe.testrunner test script). It parses the test result and exists at the end with a status code of 0 if there where no test failures at all or 1 otherwise, making it buildbot friendly. Generally it will try to run the tests for all packages in the working set and add the package to the test-path automatically. The exclude option allows to exclude packages from being tested. For example if those do not include their tests in the released distribution or may not be compatible with zope.testing. The assumption here is that you want to test everything you have with an opt-out. If some package pulls in a new dependency that dependency will be tested too. The other advanced option is to use a "groups" definition, which allows to put multiple packages into one group and then run all of them in one run (internally using ./bin/test -s foo -s bar). This is basically to avoid test layer startup cost for similar packages which are known to behave well when run together. This stuff is fairly new and might be missing features. I'd welcome feedback and patches ;) Hanno _______________________________________________ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )