[issue32317] sys.exc_clear() clears exception in other stack frames
Garrett Berg <googb...@gmail.com> added the comment:
I found a workaround, and probably the reason this has not been detected before:
import sys
def doclear2()
try:
1/0
except ZeroDivisionError:
sys.exc_clear()
try:
1/0
except ZeroDivisionError:
exc = sys.exc_info()[0]
print("third exc (before doclear):", exc)
doclear2()
exc = sys.exc_info()[0]
print("third exc (after doclear):", exc)
assert sys.exc_info()[0] is ZeroDivisionError
The above passes. It seems that being inside a local exception block allows for
the spec to hold, only the "local exception" is cleared.
I still think this is a pretty major bug, but it is good to know what the
workaround/reason it's "been working" is.
--
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32317>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32317] sys.exc_clear() clears exception in other stack frames
New submission from Garrett Berg <googb...@gmail.com>:
# Summary
In python (2 or 3) it should always be valid to have a bare *raise* statement
in an exception block.
try:
do_something()
except SomeException:
do_cleanup()
raise # always reraises SomeException with original stack trace
You can see this in the python exception documentation:
https://docs.python.org/2.7/tutorial/errors.html#raising-exceptions
# Reproduction of Failure of invariants
Example python file where this doesn't work:
from __future__ import print_function
import sys
def doclear():
"""This should basically be a noop"""
sys.exc_clear()
try:
1/0
except ZeroDivisionError:
exc = sys.exc_info()[0]
print("first exc:", exc)
assert exc is ZeroDivisionError
try:
1/0
except ZeroDivisionError:
exc = sys.exc_info()[0]
print("second exc (before doclear):", exc)
doclear()
exc = sys.exc_info()[0]
print("second exc (after doclear):", exc)
assert sys.exc_info()[0] is ZeroDivisionError # fails
Running in python2.7 gives:
first exc:
second exc (before doclear):
second exc (after doclear): None
Traceback (most recent call last):
File "doclear.py", line 23, in
assert sys.exc_info()[0] is ZeroDivisionError # fails
AssertionError
# Conclusion
There seems to be a bug in python 2.7 where it doesn't follow it's own spec.
[sys.exc_clear()](https://docs.python.org/2/library/sys.html#sys.exc_clear)
states:
> This function clears all information relating to the current or last
> exception that occurred in the current thread. After calling this function,
> exc_info() will return three None values until another exception is raised in
> the current thread or *the execution stack returns to a frame where another
> exception is being handled*.
The above code is clear example where sys.exc_clear() is changing the value of
sys.exc_info() *in a different stack frame*, which is against what is stated in
the above docs.
This bug makes it impossible to reraise with a bare raise statement when doing
cleanup that could use sys.exc_clear(). In other words, calling into functions
can *change your local state* with regards to what exception is being handled.
--
components: Interpreter Core
messages: 308264
nosy: Garrett Berg
priority: normal
severity: normal
status: open
title: sys.exc_clear() clears exception in other stack frames
type: behavior
versions: Python 2.7
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32317>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32317] sys.exc_clear() clears exception in other stack frames
Garrett Berg <googb...@gmail.com> added the comment: I forgot to post this: python --version Python 2.7.14 -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32317> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32208] Improve semaphore documentation
Garrett Berg <googb...@gmail.com> added the comment: Gave it a shot! -- keywords: +patch pull_requests: +4620 stage: needs patch -> patch review ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32208] Improve semaphore docmentation
Garrett Berg <googb...@gmail.com> added the comment: It may be desirable to change: > Once awoken, decrement the counter by 1. to > Once awoke, decrement the counter (which is guaranteed to be positive) by 1. Although this could be considered obvious since the counter can never go below zero. -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32208] Improve semaphore docmentation
Garrett Berg <googb...@gmail.com> added the comment: Also, is "The order in which threads are awoken should not be relied on" actually true? Would it not be up to the OS specific scheduler? For instance, wouldn't the POSIX scheduler use a priority queue of some kind? -- ___ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue32208> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32208] Improve semaphore docmentation
New submission from Garrett Berg <googb...@gmail.com>:
The definition of threading.Semaphore is confusing (for all versions of python
docs)
https://docs.python.org/2/library/threading.html#semaphore-objects
acquire([blocking])
It currently states the following:
> When invoked without arguments: if the internal counter is larger than zero
> on entry, decrement it by one and return immediately. If it is zero on entry,
> block, waiting until some other thread has called release() to make it larger
> than zero. This is done with proper interlocking so that if multiple
> acquire() calls are blocked, release() will wake exactly one of them up. The
> implementation may pick one at random, so the order in which blocked threads
> are awakened should not be relied on. There is no return value in this case.
However, after testing it myself I found that is missing a crutial detail.
Let's step through the docs:
> If the internal counter is larger than zero on entry, decrement it by one and
> return immediately.
This is exactly accurate and should stay the same
> If it is zero on entry, block, waiting until some other thread has called
> release() to make it larger than zero. This is done with proper interlocking
> so that if multiple acquire() calls are blocked, release() will wake exactly
> one of them up. The implementation may pick one at random, so the order in
> which blocked threads are awakened should not be relied on. There is no
> return value in this case.
This is extremely confusing and I would like to rewrite it as follows:
> If it is zero on entry block until awoken by a call to ``release()``. Once
> awoken, decrement the counter by 1. Exactly one thread will be awoken by a
> call to ``release()``. The order in which threads are awoken should not be
> relied on. ``None`` is returned in this case.
The major thing that was missing was that the **counter is decremented after
the thread is awoken**. For instance, this code *generally* passes assertions:
```
#!/usr/bin/python2
import time
from threading import Thread, Semaphore
s = Semaphore(1)
def doit():
s.acquire()
print("did it")
th1 = Thread(target=doit)
th1.start()
th2 = Thread(target=doit)
th2.start()
time.sleep(0.2)
assert not th1.is_alive()
assert th2.is_alive()
s.release()
assert s._value == 1, "The value is increased to 1 MOMENTARILY"
start = time.time()
while sum([th2.is_alive(), th3.is_alive()]) > 1:
assert time.time() - start < 0.5
time.sleep(0.1)
assert s._value == 0, "when an aquire is awoken, THEN _value is decremented"
```
Obviously this behavior should not be relied on, but it gives a picture of what
seems to be happening under the hood.
I came across this while trying to work through "The Little Book of
Semaphores". After reading these docs I mistakingly thought that they didn't
match Djestra's original semaphore since the values could not be negative. I
now realize that while they may not match that implementation under the hood,
they match it perfectly in practice since if you (for instance) ``acquire()`` 5
times and then ``release()`` 5 times the value of Semaphore._value will be the
same when all is said and done.
--
components: Library (Lib)
messages: 307536
nosy: Garrett Berg
priority: normal
severity: normal
status: open
title: Improve semaphore docmentation
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8
___
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32208>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
