[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Sam Bull]

Sam Bull  added the comment:

It has been addressed, PR should be merged this week: 
https://github.com/python/cpython/pull/28149

Like most open source projects, it just needed someone to actually propose a 
fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[James Ferguson]

James Ferguson  added the comment:

Echoing nmatravolgyi's comments - I got here after hitting this bug and I too 
am amazed it's been around so long and hasn't been addressed.  

It makes cancellation in my application very unreliable, and the reason I need 
well-controlled cancellation semantics is to allow emergency stop of physical 
movement.

--
nosy: +jamesf

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Sam Bull]

Change by Sam Bull :


--
nosy: +dreamsorcerer
nosy_count: 8.0 -> 9.0
pull_requests: +26587
pull_request: https://github.com/python/cpython/pull/28149

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Denis S. Otkidach]

Change by Denis S. Otkidach :


--
keywords: +patch
pull_requests: +24737
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26097

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Adam Liddell]

Change by Adam Liddell :


--
nosy: +aaliddell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[nmatravolgyi]

nmatravolgyi  added the comment:

I've also found this deficiency of asyncio.wait_for by debugging an obscure 
hang in an application. Back then I've quickly made an issue about it: 
https://bugs.python.org/issue43389

I've just closed it as duplicate, since this issue covers the same bug and has 
been around longer.

I'm surprised this issue has not got more attention. This definitely needs a 
fix. Ignoring a CancellationError is something that is heavily discouraged by 
the documentation in general. Silently ignoring/eating a cancellation makes 
this construct unreliable without good workarounds, aside from not using it.

For a specific application, this was so broken, that I had to come up with a 
fix in the short term. I made an asyncio.wait_for variant available as a 
library that fixes the problem: https://github.com/Traktormaster/wait-for2

The repository has a detailed description of the issue and the way it fixes it. 
It also has test cases to assert the behaviour of the builtin and the fixed 
wait_for construct from the library.

--
nosy: +nmatravolgyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Crowbar Z]

Change by Crowbar Z :


--
nosy: +crowbarz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Denis S. Otkidach]

Change by Denis S. Otkidach :


--
nosy: +ods

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

Hi Chris,
Yes, I do believe that is the respectful change, if we look the CancelledError 
is not checked to be external or originate from the timer. 
Best regards, 
Taras Voinarovskyi

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Chris Jerdonek]

Chris Jerdonek  added the comment:

It looks like issue 37658 might be the relevant change rather.

Here is the new logic it introduced: 
https://github.com/python/cpython/blob/db455296be5f792b8c12b7cd7f3962b52e4f44ee/Lib/asyncio/tasks.py#L483-L488

(via https://github.com/python/cpython/pull/21894 )

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Chris Jerdonek]

Change by Chris Jerdonek :


--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue42130] AsyncIO's wait_for can hide cancellation in a rare race condition

[Taras Voinarovskyi]

New submission from Taras Voinarovskyi :

Hi, during migration to Python 3.8.6 we encountered a behavior change from 
previous versions: wait_for ignored the request to cancellation and returned 
instead. After investigation, it seems to be related to the update in bpo-32751 
and is only reproduced if the waited task is finished when cancellation of 
wait_for happens (code mistakes external CancelledError for a timeout).

The following example can reproduce the behavior on both 3.8.6 and 3.9.0 for me:

```

import asyncio


async def inner():
return

async def with_for_coro():
await asyncio.wait_for(inner(), timeout=100)
await asyncio.sleep(1)
print('End of with_for_coro. Should not be reached!')

async def main():
task = asyncio.create_task(with_for_coro())
await asyncio.sleep(0)
assert not task.done()
task.cancel()
print('Called task.cancel()')
await task  # -> You would expect a CancelledError to be raised.


asyncio.run(main())
```

Changing the wait time before cancellation slightly will return the correct 
behavior and CancelledError will be raised.

--
components: asyncio
messages: 379454
nosy: asvetlov, tvoinarovskyi, yselivanov
priority: normal
severity: normal
status: open
title: AsyncIO's wait_for can hide cancellation in a rare race condition
type: behavior
versions: Python 3.10, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com