Re: [Python-Dev] unittest isolation and warnings

[Antoine Pitrou]

Hi Christian,

On Fri, 17 Nov 2017 10:15:24 +0100
Christian Tismer  wrote:
> 
> Example:
> One of my tests emits warnings when a certain condition is
> met. Instead of reporting the error immediately, it uses
> warnings, and at the end of the test, an error is produced
> if there were warnings.

I suggest you try using subtests.  This would allow you to report
several errors from a given test method.
https://docs.python.org/3/library/unittest.html#distinguishing-test-iterations-using-subtests

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest isolation and warnings

[Christian Tismer]

Thanks a lot!
Good to know.

Ciao -- Chris

On 17.11.17 16:44, Brett Cannon wrote:
> Tests are not isolated from the warnings system, so things will leak
> out. Your best option is to use the context manager in the warnings
> module to temporarily make all warnings raise exceptions and test for
> the exception (I'm at the airport, hence why I don't know the name of
> the context manager; the warnings module docs actually have a sample on
> how best to write tests the involve warnings).
> 
> 
> On Fri, Nov 17, 2017, 01:34 Christian Tismer,  > wrote:
> 
> Hi guys,
> 
> when writing tests, I suddenly discovered that unittest
> is not isolated to warnings.
> 
> Example:
> One of my tests emits warnings when a certain condition is
> met. Instead of reporting the error immediately, it uses
> warnings, and at the end of the test, an error is produced
> if there were warnings.
> 
>         if hasattr(__main__, "__warningregistry__"):
>             raise RuntimeError("There are errors, see above.")
> 
> By chance, I discovered that an error was suddenly triggered without
> a warning. That must mean the warning existed already from
> another test as a left-over.
> 
> My question:
> Is that known, and is that intended?
> To what extent are the test cases isolated from each other?
> 
> I do admit that my usage of warnings is somewhat special.
> But it is very convenient to report many errors on remote servers.
> 
> Cheers -- Chris
> 
> --
> Christian Tismer             :^)   tis...@stackless.com
> 
> Software Consulting          :     http://www.stackless.com/
> Karl-Liebknecht-Str. 121     :     https://github.com/PySide
> 14482 Potsdam                :     GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> 
> ___
> Python-Dev mailing list
> Python-Dev@python.org 
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
> 


-- 
Christian Tismer :^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] unittest isolation and warnings

[Brett Cannon]

Tests are not isolated from the warnings system, so things will leak out.
Your best option is to use the context manager in the warnings module to
temporarily make all warnings raise exceptions and test for the exception
(I'm at the airport, hence why I don't know the name of the context
manager; the warnings module docs actually have a sample on how best to
write tests the involve warnings).

On Fri, Nov 17, 2017, 01:34 Christian Tismer,  wrote:

> Hi guys,
>
> when writing tests, I suddenly discovered that unittest
> is not isolated to warnings.
>
> Example:
> One of my tests emits warnings when a certain condition is
> met. Instead of reporting the error immediately, it uses
> warnings, and at the end of the test, an error is produced
> if there were warnings.
>
> if hasattr(__main__, "__warningregistry__"):
> raise RuntimeError("There are errors, see above.")
>
> By chance, I discovered that an error was suddenly triggered without
> a warning. That must mean the warning existed already from
> another test as a left-over.
>
> My question:
> Is that known, and is that intended?
> To what extent are the test cases isolated from each other?
>
> I do admit that my usage of warnings is somewhat special.
> But it is very convenient to report many errors on remote servers.
>
> Cheers -- Chris
>
> --
> Christian Tismer :^)   tis...@stackless.com
> Software Consulting  : http://www.stackless.com/
> Karl-Liebknecht-Str. 121 : https://github.com/PySide
> 14482 Potsdam: GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] unittest isolation and warnings

[Christian Tismer]

Hi guys,

when writing tests, I suddenly discovered that unittest
is not isolated to warnings.

Example:
One of my tests emits warnings when a certain condition is
met. Instead of reporting the error immediately, it uses
warnings, and at the end of the test, an error is produced
if there were warnings.

if hasattr(__main__, "__warningregistry__"):
raise RuntimeError("There are errors, see above.")

By chance, I discovered that an error was suddenly triggered without
a warning. That must mean the warning existed already from
another test as a left-over.

My question:
Is that known, and is that intended?
To what extent are the test cases isolated from each other?

I do admit that my usage of warnings is somewhat special.
But it is very convenient to report many errors on remote servers.

Cheers -- Chris

-- 
Christian Tismer :^)   tis...@stackless.com
Software Consulting  : http://www.stackless.com/
Karl-Liebknecht-Str. 121 : https://github.com/PySide
14482 Potsdam: GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com