I was perusing the pytest docs and
found http://pytest.org/latest/skipping.html. It appears that in the
pytest xfail decorator one can specify the error with raises. Here's a
little test
import pytest
@pytest.mark.xfail(raises=IndexError)
def test_false_xfail():
x= [1, 2, 3]
assert x[0] == 3
@pytest.mark.xfail(raises=IndexError)
def test_true_xfail():
x= [1, 2, 3]
assert x[5] == 3
@pytest.mark.xfail(raises=IndexError)
def test_passing_xfail():
x= [1, 2, 3]
assert x[0] == 1
$ py.test test_demo.py -v
============================= test session starts
==============================
platform linux -- Python 3.4.1 -- py-1.4.25 -- pytest-2.6.3 --
/home/ptb/miniconda3/bin/python3
collected 3 items
test_demo.py::test_false_xfail FAILED
test_demo.py::test_true_xfail xfail
test_demo.py::test_passing_xfail XPASS
=================================== FAILURES
===================================
_______________________________ test_false_xfail
_______________________________
@pytest.mark.xfail(raises=IndexError)
def test_false_xfail():
x= [1, 2, 3]
> assert x[0] == 3
E assert 1 == 3
test_demo.py:6: AssertionError
================ 1 failed, 1 xfailed, 1 xpassed in 0.02 seconds
================
This would be a neat feature to add to help ensure good XFAIL tests.
On Sunday, August 24, 2014 12:16:18 PM UTC-6, Aaron Meurer wrote:
>
> That's a good idea. A good XFAIL test is different from a good normal
> test. In a good XFAIL test, you are really testing if something has started
> working at all. For example, if you want to test if integrate() can do an
> integral that it currently can't do, you should just test not
> integrate(...).has(Integral), rather than integrate(...) == answer, because
> the answer could take many forms, and the one you pick might not be the one
> that it ends up giving.
>
> Aaron Meurer
>
>
> On Sun, Aug 24, 2014 at 8:48 AM, Sergey Kirpichev <skirp...@gmail.com
> <javascript:>> wrote:
>
>> On Sat, Aug 23, 2014 at 11:36 PM, Aaron Meurer <asme...@gmail.com
>> <javascript:>> wrote:
>> > Maybe it would help to finish implementing
>> > https://github.com/sympy/sympy/issues/7044. The biggest issue with
>> XFAIL
>> > is that there will be some silly error in the way the test is written so
>> > that it never XPASSes, even when it should.
>>
>> We can add some documentation for developers ("how to write
>> XFAIL'ed tests"). The simplest case of such errors - when we can't
>> be sure in the exact form of the answer.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sympy" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to sympy+un...@googlegroups.com <javascript:>.
>> To post to this group, send email to sy...@googlegroups.com <javascript:>
>> .
>> Visit this group at http://groups.google.com/group/sympy.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/sympy/20140824134858.GA3684%40debian.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sympy+unsubscr...@googlegroups.com.
To post to this group, send email to sympy@googlegroups.com.
Visit this group at http://groups.google.com/group/sympy.
To view this discussion on the web visit
https://groups.google.com/d/msgid/sympy/67f959bd-0163-420b-876b-76ae3f812a5f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
