-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 3/22/2011 11:05 PM, Barry Warsaw wrote: > On Mar 23, 2011, at 07:31 AM, Nick Coghlan wrote: > >> On Wed, Mar 23, 2011 at 4:25 AM, Barry Warsaw <ba...@python.org> wrote: >>> It probably wouldn't be a bad idea to add a very fast "smoke test" for the >>> case where you get tripped up on the merge dance floor. When that happens, >>> you could run your localized tests, and then a set of tests that run in >>> just a >>> minute or two. >> >> What would such a smoke test cover, though? It's hard to think of >> anything particularly useful in the middle ground between "Can you run >> regrtest *at all*?" and "make quicktest". > > make quicktest runs 340 tests, and I'm certain many don't need to be run in a > smoke test. E.g. > > test_aifc > test_colorsys > test_concurrent_futures > (that's as far as it's gotten so far ;) > > Or you could time each individual test, choose a cut off for total test run > and then run whatever you can within that time (on some reference machine). > Or maybe just remove the longest running 50% of the quick tests. > > -Barry
bzr can run its 30,000 tests in about 15-30min depending on your specific hardware and platform (it also supports parallel running of the test suite). I think it takes about 1hr running single-threaded on Windows because "os.rename()" is a fairly slow operation and we do it a lot in most tests (it is part of our locking primitive). Unit-tests (tests directly on a function without setting up lots of external state) generally can execute in milliseconds. I don't specifically know what is in those 340 tests, but 18min/340 = 3.2s for each test. Which is *much* longer than simple smoke tests would have to be. John =:-> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk2J3WUACgkQJdeBCYSNAAMI2ACgl6obH9WKlmRiK4K4ib1g6SR7 KqkAn1LNrlBaUTf/sc5s30tZq3F3hmNl =DtQ8 -----END PGP SIGNATURE----- _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com