Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-15 Thread Martin v. Löwis
Collin Winter wrote: > When I've implemented this kind of thing in the past, I've generally > called the decorator/marker/whatever "TODO" (or some variation of > caps/lowercase). I usually call things TODO if they need to be done. The test case is not "TODO", since it is already done. "TODO" would

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-15 Thread Collin Winter
On 1/13/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > my main nit is the name: the test isn't broken in itself, and doesn't need > to be fixed; it's just not expected to succeed at this time. > > the usual term for this is "expected failure" (sometimes called XFAIL). > > for the developer, this me

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-14 Thread Scott David Daniels
Fred L. Drake, Jr. wrote: > Scott David Daniels wrote: > > Would "expect_fail", "expect_failure", "expected_fail", or > > "expected_failure", work for you? > > None of these use the same naming convention as the other unittest object > attributes. Perhaps something like failureExpected? > > I

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Fred L. Drake, Jr.
Scott David Daniels wrote: > Would "expect_fail", "expect_failure", "expected_fail", or > "expected_failure", work for you? None of these use the same naming convention as the other unittest object attributes. Perhaps something like failureExpected? I'd definately prefer something that reads

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Nick Coghlan
Scott David Daniels wrote: > Fredrik Lundh wrote: >> Scott David Daniels wrote: >>> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 >> my main nit is the name: the test isn't broken in itself, and doesn't need >> to be fixed; it's just not expected to succeed at this time. >> >>

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Scott David Daniels
Fredrik Lundh wrote: > Scott David Daniels wrote: >> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288 > > my main nit is the name: the test isn't broken in itself, and doesn't need > to be fixed; it's just not expected to succeed at this time. > > the usual term for this is "ex

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Scott David Daniels
OK I carried the code I offered earlier in this whole thread (tweaked in reaction to some comments) over to comp.lang.python, gathered some feedback, and put up a recipe on the cookbook. After a week or so for more comment, I'll be happy to submit a patch to include the broken_test decorator funct

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Nick Coghlan
Scott David Daniels wrote: > OK I carried the code I offered earlier in this whole thread (tweaked in > reaction to some comments) over to comp.lang.python, gathered some > feedback, and put up a recipe on the cookbook. After a week or so for > more comment, I'll be happy to submit a patch to incl

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-13 Thread Fredrik Lundh
Scott David Daniels wrote: > OK I carried the code I offered earlier in this whole thread (tweaked in > reaction to some comments) over to comp.lang.python, gathered some > feedback, and put up a recipe on the cookbook. After a week or so for > more comment, I'll be happy to submit a patch to inc

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-11 Thread Trent Mick
[Stephen J. Turnbull wrote] > > "Fredrik" == Fredrik Lundh <[EMAIL PROTECTED]> writes: > > Fredrik> many test frameworks support "expected failures" for this > Fredrik> purpose. how hard would it be to add a > > Fredrik> unittest.FailingTestCase > > Fredrik> class that r

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-11 Thread Stephen J. Turnbull
> "Fredrik" == Fredrik Lundh <[EMAIL PROTECTED]> writes: Fredrik> many test frameworks support "expected failures" for this Fredrik> purpose. how hard would it be to add a Fredrik> unittest.FailingTestCase Fredrik> class that runs a TestCase, catches any errors in it, an

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Scott David Daniels
James Y Knight wrote: > On Jan 8, 2006, at 1:01 PM, Martin v. Löwis wrote: > >> Fred L. Drake, Jr. wrote: >> >>> I like the way trial (from twisted) supports this. The test >>> method is written >>> normally, in whatever class makes sense. Then the test is marked >>> with an >>> attribute to

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Neal Norwitz
On 1/8/06, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > > many test frameworks support "expected failures" for this purpose. > how hard would it be to add a > > unittest.FailingTestCase > > class that runs a TestCase, catches any errors in it, and signals an > error ("test foo passed unexpectedly

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread James Y Knight
On Jan 8, 2006, at 1:01 PM, Martin v. Löwis wrote: > Fred L. Drake, Jr. wrote: > >> I like the way trial (from twisted) supports this. The test >> method is written >> normally, in whatever class makes sense. Then the test is marked >> with an >> attribute to say it isn't expected to pass y

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Martin v. Löwis
Fred L. Drake, Jr. wrote: > I like the way trial (from twisted) supports this. The test method is > written > normally, in whatever class makes sense. Then the test is marked with an > attribute to say it isn't expected to pass yet. When the code is fixed and > the test passes, you get that

Re: [Python-Dev] Checking in a broken test was: Re : [Python-checkins]r41940 - python/trunk/Lib/test/ test_compiler.py

2006-01-08 Thread Fred L. Drake, Jr.
On Sunday 08 January 2006 12:19, Martin v. Löwis wrote: > I don't know how hard it would be, but I would also consider this > appropriate. Of course, this should work on a case-by-case basis: > if there are multiple test methods in a class, unexpected passes > of each method should be reported.

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Martin v. Löwis
Fredrik Lundh wrote: > many test frameworks support "expected failures" for this purpose. > how hard would it be to add a > > unittest.FailingTestCase > > class that runs a TestCase, catches any errors in it, and signals an > error ("test foo passed unexpectedly") if it runs cleanly ? I don'

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Fredrik Lundh
Neal Norwitz wrote: > In the past, we haven't checked in tests which are known to be broken. > There are several good reasons for this. I would prefer you, 1) also > fix the code so the test doesn't fail, 2) revert the change (there's > still a bug report open, right?), or 3) generalize tests fo

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-08 Thread Georg Brandl
Neal Norwitz wrote: > [moving to python-dev] > >> On 1/7/06, Reinhold Birkenfeld <[EMAIL PROTECTED]> wrote: >> > Well, it is not the test that's broken... it's compiler. > > [In reference to: > http://mail.python.org/pipermail/python-checkins/2006-January/048715.html] > > In the past, we haven't

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-07 Thread Guido van Rossum
On 1/7/06, Neal Norwitz <[EMAIL PROTECTED]> wrote: > I'm proposing something like add two files to Lib/test: > outstanding_bugs.py and outstanding_crashes.py. Both would be normal > test files with info about the bug report and the code that causes > problems. I like this approach. regrtest.py wo

Re: [Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-07 Thread Tim Peters
[Neal Norwitz] > ... > In the past, we haven't checked in tests which are known to be broken. It's an absolute rule that you never check in a change (whether a test or anything else) that causes ``regretst.py -uall`` to fail. Even if it passes on your development box, but fails on someone else's

[Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py

2006-01-07 Thread Neal Norwitz
[moving to python-dev] > On 1/7/06, Reinhold Birkenfeld <[EMAIL PROTECTED]> wrote: > > Well, it is not the test that's broken... it's compiler. [In reference to: http://mail.python.org/pipermail/python-checkins/2006-January/048715.html] In the past, we haven't checked in tests which are known to