On 11/15/06, Maciek Fijalkowski <[EMAIL PROTECTED]> wrote:
> Carl Friedrich Bolz wrote:
>
> >Pierre Rouleau wrote:
> >
> >
> >>On 11/14/06, Carl Friedrich Bolz <[EMAIL PROTECTED]> wrote:
> >>
> >>
> >>>I am not quite sure I buy that argument since py.test tells you the time
> >>>that the test itself took (if you use the -v argument) or that all tests
> >>>together take (at the end of each test run). As far as I remember, these
> >>>times don't include any startup times. Yes, wall clock time of the whole
> >>>test run is quite a bad measure, but you don't have to use it anyway.
> >>>
> >>>Cheers,
> >>>
> >>>Carl Friedrich
> >>>
> >>>
> >>>
> >>I didn't realize that the time of *each* test is shown when the -v
> >>option is used. Thanks for pointing that out.
> >>
> >>
> >
> >you're welcome :-)
> >
> >Note that I didn't mean to imply that it isn't worthwhile to make
> >py.test startup time smaller. it's definitely a good thing to do, since
> >testing should have low overhead.
> >
> >Cheers,
> >
> >Carl Friedrich
> >_______________________________________________
> >py-dev mailing list
> >[email protected]
> >http://codespeak.net/mailman/listinfo/py-dev
> >
> >
> I've profiled it a bit. Right now (for me) your tests are about 3x
> slower that unittests. It still can be improved somehow.
>
> Btw: Tests are unfair, cause collection encounters 5k tests for py.test
> and 13 for unittests, if you rewrite it so they'll be equal it would be
> better. It's not good excuse anyway ;-)
>
> I'll try to work a bit more on it, but I guess achieving 2x in this
> particular case would be enough. If you still encounter massive
> slowdowns in different enviroments, do not hesitate to contact us.
>
True, the code I sent you is not comparing the exact same tests and it
is unfair.
So, as I kept looking into unit test for Python, I found nose and its
nosetests that essentially does the same as py.test (without having
the same design goals as the Py lib). I ran the same test on both
tools.
The results are:
- py.test tests in [4.30s, 4.17s, 4.28s, 4.42s]
- nosetests tests in [ 1.48s, 1.43s, 1.40s, 1.50s]
So it looks like achieving a 2x would make it run in the same ballpark
as nosetests (at least for that code).
I do like py.test error report better than nosetest though. The
reporting is less verbose.
py.test::
_______________________________________________________________________________
________________________ entrypoint: test_bad_roman[0]
________________________
def check(exception_cls, invalid_numeral):
print 'Testing : ', invalid_numeral
E raises(exception_cls, roman.fromRoman, invalid_numeral)
> ExceptionFailure: DID NOT RAISE
[D:\dev\python\test\test_roman.py:164]
- - - - - - - - - - - - - - [0]: recorded stdout - - - - - - - -
- - - - - - -
Testing : MMMM
_______________________________________________________________________________
============ tests finished: 5023 passed, 1 failed in 4.17 seconds
============
and the same failure shown by nosetest::
======================================================================
ERROR: test_roman.test_bad_roman:(<class
roman.InvalidRomanNumeralError at 0x00C5DDB0>, 'MMMM')
----------------------------------------------------------------------
Traceback (most recent call last):
File "c:\python24\lib\site-packages\nose-0.9.1-py2.4.egg\nose\case.py",
line 52, in runTest
self.testFunc()
File "c:\python24\lib\site-packages\nose-0.9.1-py2.4.egg\nose\loader.py",
line 324, in run
expr[0](*expr[1:])
File "D:\dev\python\test\test_roman.py", line 164, in check
raises(exception_cls, roman.fromRoman, invalid_numeral)
File "C:\py-dist\py\test\raises.py", line 39, in raises
expr=args, expected=ExpectedException)
ExceptionFailure: DID NOT RAISE
-------------------- >> begin captured stdout << ---------------------
Testing : MMMM
--------------------- >> end captured stdout << ----------------------
----------------------------------------------------------------------
Ran 5024 tests in 1.407s
FAILED (errors=1)
Cheers,
P.R.
_______________________________________________
py-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/py-dev
