Michael Foord added the comment:
Assertions are not uncommon in setUp. setUp is for setting up common state
shared between tests and I regularly want to assert that state creation /
preconditions are correct.
I've never been bitten by this issue (I rarely use expectedFailure), but it's
R. David Murray added the comment:
Can you ever imagine the assertions in the setUp being what you would want
reported as an expected failure? I would think that setUp assertion failure
would be something you would want to be always reported as a failure, even if
you expect the test itself
Michael Foord added the comment:
Maybe if the expectedFailure is applied to the whole class and it's the setUp
that is unable to work. I've never seen it used that way of course (mostly
because it doesn't work that way) - but I *can* imagine it.
--
Robert Collins added the comment:
assertions in setUp are fine IMO. But here's the thing. WHat should this code
do?
class Demo(unittest.TestCase):
def setUp(self):
raise Exception('hi')
def test_normal(self):
# this should NOT be covered by expectedFailure
Nick Coghlan added the comment:
If you want the simple model, there's nothing to change (except perhaps
documentation), as that's the way things work today.
subtest support also makes it much easier to factor out common assertions
without relying on the setUp/tearDown protocol, so I'd be OK
R. David Murray added the comment:
Oh, now I understand what the issue is. It never even *occurred* to me that
someone might put assertions in setUp or tearDown. So, yeah, count me in on
considering that an abuse of the protocol :)
(If I want to factor out common assertions, and I do it a
Nick Coghlan added the comment:
Heh, yeah, I've written *many* check_* methods in my life - I just hadn't made
the link between that practice, and this suggestion.
I think that switches me firmly to the don't change the behaviour camp.
--
___
Robert Collins added the comment:
Just to note that unittest2 tip (unreleased)had michaels proposed fix, which is
different to that here. I'm going to back that out before doing a release,
because they should be harmonisious.
--
nosy: +rbcollins
Robert Collins added the comment:
My take on this, FWIW, is that any methods in the under-test API - setUp,
tearDown, test_* and anything registered via addCleanup should all support the
same protocol as much as possible, whatever it is.
That is, raising a skip in setUp should skip the test.
Nick Coghlan added the comment:
While I agree with Robert's comments in general, the main question to be
resolved here is the scope of the expectedFailure decorator.
Yes, it's applied specifically to the test execution phase when writing the
code, but the question is how the following
Robert Collins added the comment:
I'd argue for
- An error - its not covered by the decorator.
- An error - the last two are not covered by the decorator.
- An error - the last two are not covered by the decorator.
- An error - the exception isn't a 'failure' [today - we should revisit this
R. David Murray added the comment:
I agree with Robert. I'd rather get notified of a failure in the cleanup...if
the test is an *expected* failure, then its cleanup should be designed to
succeed when the test fails; anything else would be a bug in the design of the
test, IMO.
--
Nick Coghlan ncogh...@gmail.com added the comment:
As another data point, this question came up again in the context of issue
#12958.
The new test_socket.ThreadableTest uses tearDown() to pick up and reraise any
exception that occurred in the client thread. This meant that my initial
Michael Foord mich...@voidspace.org.uk added the comment:
I think Twisted uses the tearDown to fail tests as well. As we have two use
cases perhaps we should allow expectedFailure to work with failues in tearDown?
(And if we do that it should cover setUp as well for symmetry or it becomes a
Michael Foord mich...@voidspace.org.uk added the comment:
Well, the original report is here:
http://code.google.com/p/unittest-ext/issues/detail?id=21
I copied all the details provided into this issue though. Obviously the
original reporter feels that they have a genuine use case.
There is
holger krekel holger.kre...@gmail.com added the comment:
Michael, if you have it i'd like to see the original post/concrete use case.
thanks, holger
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10548
Changes by holger krekel holger.kre...@gmail.com:
--
nosy: +hpk
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10548
___
___
Python-bugs-list
R. David Murray rdmur...@bitdance.com added the comment:
I would expect this code to report an error of some sort, not pass as an
expected failure. The expected failure should be in the test case *only*, not
in the setup or teardown methods. That is, I don't think this is a bug, I
think it
holger krekel holger.kre...@gmail.com added the comment:
FWIW i tend to agree and would probably prefer setup/teardown to result in an
error rather than be subsumed in an expected-to-fail marked test. I guess if
one regards setup/teardown as a place to implement pre/post-conditions than the
Michael Foord mich...@voidspace.org.uk added the comment:
(made slightly redundant by Holger's comment but I'll continue anyway)
I think the issue is that setUp / tearDown are used for two different purposes.
The first is setting up and tearing down test infrastructure - where you do
want to
R. David Murray rdmur...@bitdance.com added the comment:
I have to say that it would never have occurred to me to assert a pre or post
condition and an expected failure where I expected the pre or post condition to
fail, but if you've got a real use case and it would make the code simpler, I
Michael Foord mich...@voidspace.org.uk added the comment:
The same is also true for tearDown and cleanUp functions.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue10548
___
Michael Foord mich...@voidspace.org.uk added the comment:
Note that if an error is raised in a tearDown or cleanUp then
unexpected-success should not be reported either. Not very important but might
as well be fixed at the same time.
--
___
Python
New submission from Michael Foord mich...@voidspace.org.uk:
Reported by Konrad Delong.
class MyTest(unittest.TestCase):
def setUp(self):
raise Exception
@unittest.expectedFailure
def testSomething(self):
assert False, test method failed
This code will report
24 matches
Mail list logo
