On Wed, May 29, 2013 at 16:26 -0400, Daniel Holth wrote: > On Wed, May 29, 2013 at 3:33 PM, holger krekel <hol...@merlinux.eu> wrote: > > On Wed, May 29, 2013 at 15:15 -0400, Daniel Holth wrote: > >> On Wed, May 29, 2013 at 2:36 PM, holger krekel <hol...@merlinux.eu> wrote: > >> > Hi Nick, > >> > > >> > On Mon, May 27, 2013 at 21:36 +1000, Nick Coghlan wrote: > >> >> After preliminary reviews by Donald and Daniel, I have now pushed the > >> >> first complete draft of the JSON-based metadata 2.0 proposal to > >> >> python.org > >> >> > >> >> PEP 426 (metadata 2.0): http://www.python.org/dev/peps/pep-0426/ > >> > > >> > After a first quick read i am wondering if i missed something > >> > with respect to test tools. There are some fields which specify > >> > dependencies required for running tests, but there is nothing that > >> > would tell which test runner to use, which test command to invoke, > >> > or am i missing something? > >> > > >> > Basically i am wondering how PEP426 could be useful/used by tox. > >> > best, > >> > >> The first thing we might do is to have setuptools expose its > >> test_suite argument as "extensions" : { "setuptools": { "test_suite": > >> "callable.name" } }. > > > > The way tox specifies testing is to allow arbitrary test commands not > > just unittest test suites. People use py.test, make and whatnot > > to run tests. > > tox could shove a JSON version of its config into the metadata as an > extension. > > If "something wrapped in a unittest suite" isn't the interface for > running tests (in the most boring way possible, with better runners > for the developers not the users of the package), then what should it > be?
An entry point or way to execute a command line script, either coming from an test_requires dependency, or a script contained in the package. Note that py.test can run unittest test suites, but pytest test suites are not modeled as a unittest test suite. The same would be true if you use a test script contained in a package or "make" for that matter. IOW, i think there should be a field denoting which command line tool or script to run for the tests. If it's not defined and a test suite is defined, it's of course fine to invoke unitest's machinery on it. > >> I think we need the next version of the metadata or sdist 2.0 to > >> really do a better job than just running "setup.py test"; right now > >> the tests usually only make sense in the context of an unpacked sdist. > >> Input appreciated. > > > > We certainly don't want to advocate using "setup.py" for running tests as > > we want to get way from the neccessity for this file to exist (correct me > > if i am wrong with this presumption). The metadata should be rich > > enough to support tools like tox to perform the testing, much > > like pip for installations. > > The first priority is to get rid of "setup.py install" since that is > where most of the trouble comes from. It's not that big of a task to > relegate setup.py to its proper place as a build script. At the same > time it will eventually become possible to use non-distutils-derived > build tools without having to emulate a distutils setup.py. > > Wouldn't we continue to type tox, or nose, or py.test, with the > standard hook being mainly for automated tools that just want to > verify that something still works before building / installing? Typically projects depend on the test runner/framework for executing their tests, be it at development or pre-installation time. holger _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig