Phillip J. Eby wrote: > At 08:46 AM 1/5/2007 -0500, Jim Fulton wrote: > >> Here is a rough draft proposal for declaring tests in eggs: >> >> Introduction >> ============ >> >> Software packages should have automated tests. Consumers of >> packages will often want to run these tests. Tools should be able to >> do this automatically. This proposal seeks to provide a way for >> automated tools to discover tests in distributions, including eggs, so >> that tests can be run or so that test runners can be automatically >> created to run the tests. >> >> Proposal >> ======== >> >> This proposal aims to be extremely simple. It has 2 parts: >> >> 1. A 'test_suite' entry point is defined. An egg can provide zero or >> more test_suite entry points. These entry points will define >> callable objects that can be called without arguments and that >> return unittest test suites. >> >> 2. An optional 'tests' extra is defined. When creating test runners >> or dynamically loading distributions to load tests, any >> distributions listed in extra requires for the 'tests' extra shall >> be included in the working set for the test runner. >> >> Thoughts? > > Point #2 is unnecessary, since individual entry points can list extras > in square brackets following the module/attribute information. When > loading an entry point, these extras automatically get require()'d. > Conversely, if the test runner wants to manage the loading process, it > can simply inspect the entry point object to determine the names of the > extras.
Ah, good point. Man, setuptools is soooooo rich. > Regarding point #1, I'm not sure this is enought to define what's > necessary. For example, it should perhaps be stated that the entry > point must be in code that will be installed by either the distribution > itself, or that is included in the code provided by the entry point's > extras. OK > Also, an egg can't provide more than one entry point with the same name, > so rather than a 'test_suite' entry point, we probably want an entry > point *group*, perhaps something like 'installed.test_suites'. That's what I meant to say. I'll clarify. > Next, there needs to be reasonable support for dynamic test discovery, > such as setuptools' own ScanningLoader. I would suggest that there be > an optional entry point for the test loader class to be used to process > the other entry points' target objects. The standard unittest protocol > for loadTestsFromName() takes two arguments: a name and an object. > Passing an empty string for the name, and the object loaded by the test > suite entry point, suffices to enable normal behavior for loaders such > as ScanningLoader, and I believe 'nose' as well as any other > well-behaved unittest extensions. > > However, I think the spec should try to define what "well-behaved" means > in terms of I/O, result reporting, etc. (Obviously, one requirement is > that the test loader must be able to take an empty name string and an > object in its loadTestsFromName() method, and return a test suite.) Why can't an entry point invoke a test loader itself? This seems much simpler and more straightforward to me. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
