[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

Hi Chris,
Yes, I do believe that is the respectful change, if we look the CancelledError 
is not checked to be external or originate from the timer. 
Best regards, 
Taras Voinarovskyi

--

___
Python tracker 
<https://bugs.python.org/issue42130>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Taras Voinarovskyi]

New submission from Taras Voinarovskyi :

Hi, during migration to Python 3.8.6 we encountered a behavior change from 
previous versions: wait_for ignored the request to cancellation and returned 
instead. After investigation, it seems to be related to the update in bpo-32751 
and is only reproduced if the waited task is finished when cancellation of 
wait_for happens (code mistakes external CancelledError for a timeout).

The following example can reproduce the behavior on both 3.8.6 and 3.9.0 for me:

```

import asyncio


async def inner():
return

async def with_for_coro():
await asyncio.wait_for(inner(), timeout=100)
await asyncio.sleep(1)
print('End of with_for_coro. Should not be reached!')

async def main():
task = asyncio.create_task(with_for_coro())
await asyncio.sleep(0)
assert not task.done()
task.cancel()
print('Called task.cancel()')
await task  # -> You would expect a CancelledError to be raised.


asyncio.run(main())
```

Changing the wait time before cancellation slightly will return the correct 
behavior and CancelledError will be raised.

--
components: asyncio
messages: 379454
nosy: asvetlov, tvoinarovskyi, yselivanov
priority: normal
severity: normal
status: open
title: AsyncIO's wait_for can hide cancellation in a rare race condition
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue42130>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

For now, it seems like ``copy.copy(exception)`` before raising seems to prevent 
this behaviour. So for all exceptions originating from background tasks I raise 
a copy to the user, rather than the original exception. It prints correct 
stack, so no real impact on the library.

--

___
Python tracker 
<https://bugs.python.org/issue35751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

So yes, the `clear_frames` function will force a running generator to close. 
See https://github.com/python/cpython/blob/3.7/Objects/frameobject.c#L566, it 
explicitly does a Finalize. Would that be the desired behaviour for 
assertRaises is not clear. I find it strange that catching an exception is 
closing my running coroutine.

The reproduce example can be lowered to something like::

import asyncio


async def background(error_future):
try:
raise ValueError
except Exception as exc:
error_future.set_exception(exc)

await asyncio.sleep(1)


async def main():
loop = asyncio.get_event_loop()
error_future = loop.create_future()
task = asyncio.create_task(background(error_future))

await asyncio.wait([error_future])
exc = error_future.exception()
import traceback
traceback.clear_frames(exc.__traceback__)

# Will block forever, as task will never be waken up
await task


if __name__ == "__main__":
asyncio.run(main())

--

___
Python tracker 
<https://bugs.python.org/issue35751>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com