This isn't too different from what I have in mind, except that my main
testrunner.py script uses introspection to discover the tests. Adding a
new test set won't require editing testrunner at all. The testrunner
will also filter the tests according to certain criteria, such as mpm or
os, so that tests for a specific platform are automatically selected.
More later.
Jim
Mike Looijmans wrote:
What I have been doing in a totally unrelated Python project is to
create test groups simply by putting them into separate modules. The
main test module test.py looks like this:
## (test.py)
import unittest
from test_something import *
from test_someother import *
from test_yetmore import *
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
This works because unittest takes all 'things' that start with 'test' in
the provided module and runs them. So anything we bring into our
namespace gets run. This also makes it possible to import tests from
other projects, and share these tests between projects.
The other test_ modules look much alike:
## (test_something.py)
import unittest
import test_peer
class test04MultiPeerSystem(test_peer.BaseTestRealThing):
def test03DiscoveryC08B20(self):
"Multiple clients w/ discovery: 8 peers, 20 blocks each"
self.runDiscoveryTest(nclients = 8, nblocks=20)
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
This makes it very easy to handle test subsets, and run single test
suites. Just run
$python test.py
to run ALL the tests. To run just a single set, run
$python test_something.py
And to run a single test, either of these will do:
$python test_something.py test04MultiPeerSystem
$python test.py test04MultiPeerSystem
The real power shows when you want to run 4 or 5 test sets, and/or only
parts of some test sets. Just create a new "main" test unit that imports
the desired ones, and you're set:
## (test_few.py)
import unittest
from test_something import *
from test_someother import TestOnlyThis
if __name__ == '__main__':
unittest.main(module=__import__(__name__))
##
Because some tests take very long to run (in my vocabulary, "long" means
more than a second), this saves me a lot of time when working on a part
of a big project, where I don't need to run all tests all the time.
--
Mike Looijmans
Philips Natlab / Topic Automation
Jim Gallacher wrote:
...
I've been playing with some ideas for a new test framework, using a
subclass of unittest.TestLoader to find and configure tests. I want to
play around with it for another day or so before sharing but at this
point I'm pretty confident it'll work.
Creating a new set of tests could be as simple as:
testsuites/core/simpletest.py
----------------------------
from testlib import VirtualHostTest
class MyTests(VirtualHostTest):
def test_hello_world(self):
rsp = self.send_request()
self.failUnless(rsp == 'test ok')
def test_goodbye_world(self):
rsp = self.send_request()
self.failUnless(rsp == 'test ok')
htdocs/testhandlers/core/simpletest.py
--------------------------------------
from mod_python import
def test_hello_world(req):
req.write('test ok')
return apache.OK
def test_goodbye_world(req):
req.write('test ok')
return apache.OK
$ python testrunner.py
Things like virtual host names and handler directives required for
configuration or send_request() are automatically derived from the
test class and test method names. It will still be possible to provide
custom apache configuration directives in a manner similar to that
which we currently use, but for most tests this will not be required.
Usage would look something like this:
Run all the tests
$ python testrunner.py
Run one test
$ python testrunner.py -t core.simpletest.MyTests.test_hello_world
Run a group of tests (this would load the TestCase subclasses in
testsuites/sessions/filesession.py):
$ python testrunner.py -t sessions.filesession
Jim