New submission from nmatravolgyi <[email protected]>:
I have found myself debugging a *very* not intuitive behavior regarding
asyncio.wait_for that I'd consider a bug/deficiency. The problem very simply
put: wait_for will return the wrapped future's result even when it is being
cancelled, ignoring the cancellation as it has never existed.
This will make parallel execution-waits hang forever if some simple conditions
are met. From the perspective of this snippet every task must exit so it just
needs to wait. I know cancellation *can* be ignored, but it is discouraged by
the documentation for this reason exactly.
tasks = [...]
for t in tasks:
t.cancel()
results = await asyncio.gather(*tasks, return_exceptions=True)
I already know that this behavior has been chosen because otherwise the
returned value would be lost. But for many applications, losing an explicit
cancellation error/event is just as bad.
The reason why ignoring the cancellation is critical is because the cancelling
(arbiter) task cannot reliably solve it. In most cases having repeated
cancellations in a polling wait can solve this, but it is ugly and does not
work if the original wait_for construct is in a loop and will always ignore the
cancellation.
The most sensible solution would be to allow the user to handle both the return
value and the cancellation if they do happen at once. This can be done by
subclassing the CancelledError as CancelledWithResultError and raising that
instead. If the user code does not handle that exception specifically then the
user "chose" to ignore the result. Even if this is not intuitive, it would give
the user the control over what really is happening. Right now, the user cannot
prefer to handle the cancellation or both.
Lastly, I may have overlooked something trivial to make this work well. Right
now I'm considering replacing all of the asyncio.wait_for constructs with
asyncio.wait constructs. I can fully control all tasks and cancellations with
that. I've made a simple demonstration of my problem, maybe someone can shed
some light onto it.
----------
components: asyncio
files: aio_wait_for_me.py
messages: 388032
nosy: asvetlov, nmatravolgyi, yselivanov
priority: normal
severity: normal
status: open
title: Cancellation ignored by asyncio.wait_for can hang application
type: behavior
versions: Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49849/aio_wait_for_me.py
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue43389>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
