[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

No problem, after some debugging the statement finally executed is 
'exec(compile("set.add(0)", "foo.py", "exec"))' . In Python interpreter this 
will give the expected error. It also gives the same error before 
coverage.start() [0] and gives a different one as in the report once the 
statement is executed after coverage.start(). So this might be something to do 
with coverage and I have less knowledge about the internals.

I am closing this as third party since I couldn't see any issue with CPython 
and you might want to follow up coverage issue tracker. Thanks for the report 
though!

[0] 
https://github.com/nedbat/coveragepy/blob/f12db7fd4b3ab288f7f770f7138062951feaa6c9/coverage/cmdline.py#L639

--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[SylvainDe]

SylvainDe  added the comment:

I have no explanation whatsoever but removing the call to coverage seems to 
remove the issue: 
https://travis-ci.org/SylvainDe/DidYouMean-Python/builds/491727724 .

I guess we can close this issue - sorry for the inconvenience.

@nedbat: do you want me to open an issue in coverage ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[SylvainDe]

SylvainDe  added the comment:

This is a brilliant idea! I'll give it a try and keep you posted.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Is this something related to running under coverage? With the failure commit on 
Travis I built the binary and there are no failures with unittest but the tests 
fail with coverage. Running 'set.add(0)' under coverage and as normal file 
generates different error messages. Can you try running with unittest to see if 
there are any discrepancies? I am adding @nedbat who might have a better 
explanation.

# Python version

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python
Python 3.8.0a1+ (tags/v3.8.0a1-23-g8a03ff2ff4:8a03ff2ff4, Feb 11 2019, 15:51:54)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

# set.add(0) under coverage and as a file

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ echo "set.add(0)" > 
foo.py
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ coverage run foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in 
set.add(0)
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python foo.py
Traceback (most recent call last):
  File "foo.py", line 1, in 
set.add(0)
TypeError: descriptor 'add' requires a 'set' object but received a 'int'

# Running the reported case will fail while running under coverage

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ python -m unittest 
discover --start-directory=/tmp/suggest --pattern=*.py
..
--
Ran 2 tests in 0.002s

OK
(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ coverage run -m 
unittest discover --start-directory=/tmp/suggest --pattern=*.py
FF
==
FAIL: test_assertRaisesRegex (foo.FooTest)
--
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/suggest/foo.py", line 8, in test_assertRaisesRegex
self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE, set.add, 0)
AssertionError: "descriptor '\w+' requires a 'set' object but received a 'int'" 
does not match "descriptor 'add' for 'set' objects doesn't apply to 'int' 
object"

==
FAIL: test_assertRaisesRegex_contextman (foo.FooTest)
--
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/tmp/suggest/foo.py", line 12, in test_assertRaisesRegex_contextman
set.add(0)
AssertionError: "descriptor '\w+' requires a 'set' object but received a 'int'" 
does not match "descriptor 'add' for 'set' objects doesn't apply to 'int' 
object"

--
Ran 2 tests in 0.003s

FAILED (failures=2)


# Executed test

(foo-venv) karthi@ubuntu-s-1vcpu-1gb-blr1-01:~/cpython$ cat /tmp/suggest/foo.py
import unittest

DESCRIPT_REQUIRES_TYPE_RE = r"descriptor '\w+' requires a 'set' object but 
received a 'int'"

class FooTest(unittest.TestCase):

def test_assertRaisesRegex(self):
self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE, set.add, 0)

def test_assertRaisesRegex_contextman(self):
with self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE):
set.add(0)

if __name__ == "__main__":
unittest.main()

--
nosy: +nedbat, xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[SylvainDe]

SylvainDe  added the comment:

That's the very weird thing. I've been unable to reproduce this locally but it 
can be seen on Travis runs.

Here are the results so far (Versions numbers are retrieved with python -VV and 
python -c "import sys; print(sys._git)"):

On some versions, both tests pass:
Python 3.6 and before
Python 3.7.0a4+ (heads/master:4666ec5, Jan 26 2018, 04:14:24) - [GCC 
4.8.4] - ('CPython', 'heads/master', '4666ec5'))

On some versions, both tests fail:
Python 3.8.0a1+ (heads/master:8a03ff2, Feb 9 2019, 07:30:26) [GCC 5.4.0 
20160609] - ('CPython', 'heads/master', '8a03ff2')

On some versions, only test_assertRaisesRegex_contextman fails which is 
what I find confusing:
Python 3.7.1 (default, Dec 5 2018, 18:09:53) [GCC 5.4.0 20160609] - 
('CPython', '', '')
Python 3.7.2+ (heads/3.7:3fcfef3, Feb 9 2019, 07:30:09) [GCC 5.4.0 
20160609] - ('CPython', 'heads/3.7', '3fcfef3')


Corresponding Travis build: 
https://travis-ci.org/SylvainDe/DidYouMean-Python/builds/491118939 .
Corresponding code on Github: 
https://github.com/SylvainDe/DidYouMean-Python/blob/issue36_investigation/didyoumean/didyoumean_sugg_tests.py

I'm happy to add more debug information and retrigger builds if needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[Ronald Oussoren]

Ronald Oussoren  added the comment:

On which Python versions do you see this problem? Both tests pass on my system 
(macOS 10.4.3) with Python 3.7.2, 3.6.3 and 3.5.5.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35965] Behavior for unittest.assertRaisesRegex differs depending on whether it is used as a context manager

[SylvainDe]

New submission from SylvainDe :

On some Python versions, the following pieces of code have a different behavior 
which is not something I'd expect:

```

DESCRIPT_REQUIRES_TYPE_RE = r"descriptor '\w+' requires a 'set' object but 
received a 'int'"

...

def test_assertRaisesRegex(self):
self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE, set.add, 0)

def test_assertRaisesRegex_contextman(self):
with self.assertRaisesRegex(TypeError, DESCRIPT_REQUIRES_TYPE_RE):
set.add(0)

```

On impacted Python versions, only test_assertRaisesRegex_contextman fails while 
test_assertRaisesRegex works fine.

Logs for the failure:


```

==
FAIL: test_assertRaisesRegex_contextman 
(didyoumean_sugg_tests.SetAddIntRegexpTests)
--
TypeError: descriptor 'add' for 'set' objects doesn't apply to 'int' object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/build/.../didyoumean/didyoumean_sugg_tests.py", line 23, 
in test_assertRaisesRegex_contextman
set.add(0)
AssertionError: "descriptor '\w+' requires a 'set' object but received a 'int'" 
does not match "descriptor 'add' for 'set' objects doesn't apply to 'int' 
object"


```


Either I am missing something or it looks like a bug to me.

If needed, more details/context can be found on the StackOverflow question I 
opened: 
https://stackoverflow.com/questions/54612348/different-error-message-when-unittest-assertraisesregex-is-called-as-a-context-m
 .
I can provide the details directly here if it is relevant.

--
components: Tests
messages: 335212
nosy: SylvainDe
priority: normal
severity: normal
status: open
title: Behavior for unittest.assertRaisesRegex differs depending on whether it 
is used as a context manager
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com