[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Robert Collins  added the comment:

Oh, I didn't mean to imply that these are the only options I'd support - just 
that these are the things I've thought through and that I think will all work 
well... I'm sure there are more options available ;)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Change by Robert Collins :


--
versions: +Python 3.9 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Robert Collins  added the comment:

Ok so design wise - there is state on the TestCase that influences assertions; 
in potentially two ways.

The first way is formatting - the amount of detail shown in long list 
comparisons etc.

The second way I don't think we have at the moment, but potentially it could 
influence the fidelity of comparisons for NearlyEquals and the like - generally 
though we tend to pass those in as parameters.

So just ripping everything off into standalone functions loses the home for 
that state. It either becomes global (ugh), or a new object that isn't a test 
case but is basically the same magic object that has to be known is carried 
around, or we find some other way of delegating the formatting choice and 
controls.

hamcrest has some prior art in this space, and testtools experimented with that 
too. wordpress has managed to naff up the formatting on my old blog post about 
this 
https://rbtcollins.wordpress.com/2010/05/10/maintainable-pyunit-test-suites/ 
and https://testtools.readthedocs.io/en/latest/for-test-authors.html#matchers

Its been on my TODO for a very long time to put together a PEP for adding 
matchers to the stdlib; I find the full system we did in testtools very useful 
because it can represent everything from a trivial in-memory string error 
through to a disk image for a broken postgresql database, without running out 
of memory or generating mojibake but if we wanted to do something smaller 
that didn't prejuidice extensions like testtools still doing more that would be 
fine too.

The core idea of matchers is that rather than a standalone function f() -> 
nil/raise, you build a callable object f() -> Option(Mismatch), and a Mismatch 
can be shown to users, combined with other mismatches to form composites or 
sequences and so forth. So this would give room for the state around object 
formatting and the like too.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[R. David Murray]

R. David Murray  added the comment:

"But - what are we solving for here?"  I'll tell you what my fairly common use 
case is.  Suppose I have some test infrastructure code, and I want to make some 
assertions in it.  What I invariably end up doing is passing 'self' into the 
infrastructure method/class just so I can call the assert methods from it.  I'd 
much rather be just calling the assertions, without carrying the whole test 
object around.  It *works* to do that, but it bothers me every time I do it or 
read it in code, and it makes the infrastructure code needlessly more 
complicated and slightly harder to understand/read.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Pascal Chambon]

Pascal Chambon  added the comment:

"Suppose failureException is set to TypeError on that TestCase class, how would 
your assertEquals signal failure to the test runner?"

failureException is an artefact from unittest.TestCase. It's only supposed to 
be used in a TestCase context, with an unittest-compatible runner. If people 
corrupt it, I guess it's their problem?

The point of decoupling is imho that other test runner might use the separate 
set of assertions. These assertions should raise a sensible default (i.e 
AssertionError) when encountering troubles, and accepting an alternate class as 
parameter will allow each test framework to customize the way these assertions 
behave for it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Robert Collins  added the comment:

Right now that attribute could be set by each test separately, or even varied 
within a test.

TBH I'm not sure that the attribute really should be supported; perhaps 
thinking about breaking the API is worth doing.

But - what are we solving for here. The OP here seems interested in using the 
assertion like things entirely outside of a test context.

What would a nice clean API for that be? (Yes I like matchers, but put that 
aside - if the APIs aren't close enough, lets make sure we do a good job for 
each audience rather than a compromise..)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Michael Foord]

Michael Foord  added the comment:

Or even making the assert methods into custom descriptors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Michael Foord]

Michael Foord  added the comment:

Hmm, it could be done by __init_subclass__ potentially.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Michael Foord]

Michael Foord  added the comment:

Suppose failureException is set to TypeError on that TestCase class, how would 
your assertEquals signal failure to the test runner?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Pascal Chambon]

Pascal Chambon  added the comment:

I don't get it, why would failureException block anything ? The 
unittest.TestCase API must remain the same anyway, but it could become just a 
wrapper towards external assertions.

For example :

class TestCase:

   assertEqual = wrap(assertions.assert_equal)

Where "wrap" for example is some kind of functools.partial() injecting into 
external assertions a parameter "failure_exception_class". Having all these 
external assertions take such a parameter (defaulting to AssertionError) would 
be a great plus for adaptability anyway.

--
versions: +Python 3.5 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Michael Foord]

Michael Foord  added the comment:

Has anyone seen any real world use cases for failureException? It's a real 
hindrance to a whole bunch of changes sounds decoupling. 

On the other hand something like assertThat could catch a custom exception from 
the matchers (subclass of AssertionError) and raise failureException

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Pascal Chambon]

Pascal Chambon  added the comment:

(Redirected here from https://bugs.python.org/issue37262)

I haven't dug the assertThat() idea, but why not make, as a first step, turn 
assertion methods in TestCase to staticmethods/classmethods, instead of 
instance methods?

Since they (to my knowledge) don't need to access an instance dict, they could 
be turned into such instance-less methods, and thus be usable from other 
testing frameworks (like pytest, for those who want to use pytest fixtures and 
yet benefit from advanced assertions like Django's TestCase's assertions).

"failureException" and others are meant to be (sub)class attributes, so no 
backwards incompatible change should occur (unless someone did really weird 
things with manually instantiated TestCases).

--
nosy: +pakal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Robert Collins  added the comment:

Sorry for the slow reply here;

There are API breaks involved in any decoupling that involves the exception 
raising because of the failureException attribute. Something with signalling 
that can be adapted by other test suites etc might have merit, but I think we 
are lacking a clear use case for doing this to the existing exceptions. Setting 
up a way for new things to be more easily used by users of other test 
frameworks is quite attractive; perhaps just writing them as separate functions 
with an adapter to failureException would be sufficient.

--
versions: +Python 3.9 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Robert Collins]

Robert Collins added the comment:

Hi, I'm glad you're interested in this. I very much want to see a 
matcher/hamcrest approach rather than a library of assertions per se - because 
match-or-except makes layering things harder.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Here are most popular idioms which deserve special assertion methods:

assertHasAttr(obj, name) == assertTrue(hasattr(obj, name))
assertIsSubclass(type, expected) == assertTrue(issubclass(type, expected))
assertTypeIs(obj, expected) == assertIs(type(obj), expected)
assertTypedEqual(actual, expected) == assertIs(type(actual), type(expected)) 
and assertEqual(actual, expected) # or assertIsInstance(actual, type(expected))?
assertStartsWith(actual, prefix) == assertTrue(s.startswith(prefix))
assertEndsWith(actual, suffix) == assertTrue(s.endswith(suffix))
assertUnorderedSequenceEqual(first, second) == assertTrue(all(x in second for x 
in first)) and assertTrue(all(x in first for x in second)) and 
assertEqual(len(first), len(second))

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Martin Panter]

Changes by Martin Panter vadmium...@gmail.com:


--
nosy: +vadmium

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Julian Berman]

Changes by Julian Berman julian+python@grayvines.com:


--
nosy: +Julian

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Éric Araujo]

Changes by Éric Araujo mer...@netwok.org:


--
nosy: +eric.araujo

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[Gregory Salvan]

Gregory Salvan added the comment:

issue18054 :

- adding assertCleanError in the ipaddress module, 
- suggesting assertCleanTraceback, assertRaisedFrom in unittest 

- usefull but TestCase has already a wide api.

A solution like Testtools assertThat with matcher protocol 
(https://testtools.readthedocs.org/en/latest/for-test-authors.html#matchers) 
would not expand too much TestCase api and permit to easily extend assertions.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue19645] decouple unittest assertions from the TestCase class

[R. David Murray]

R. David Murray added the comment:

You should probably start by summarizing the assertThat proposal from issue 
18054, which suggested making a separate issue for assertThat.

--
nosy: +r.david.murray
title: Improving unittest assertions - decouple unittest assertions from the 
TestCase class

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue19645
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com