New submission from Alex DeLorenzo :
According to the documentation, asyncio.as_completed() takes a positional
argument, aws, as an iterable of awaitables[1]:
asyncio.as_completed(aws, *, loop=None, timeout=None)
Run awaitable objects in the aws iterable concurrently.
As seen in the attached as_completed_gen.py file, built-in containers like
lists, and iterators over them, are accepted by as_completed() as the first
parameter without raising an error.
However, as_completed() raises TypeError if it is called with an iterable of
awaitables that is also a generator. There are examples of this behavior in
as_completed_gen.py, but here is a short example using a generator expression
in the main() coroutine function:
from asyncio import run, as_completed
async def example(): pass
async def main():
coros = (example() for _ in range(10))
for coro in as_completed(coros): # raises TypeError
await coro
run(main())
Running that example will raise a TypeError with this message:
TypeError: expect an iterable of futures, not generator
If we look at the first line in the body of as_completed(), we can see why this
error is thrown for generators that yield awaitables:
def as_completed(fs, *, loop=None, timeout=None):
if futures.isfuture(fs) or coroutines.iscoroutine(fs):
raise TypeError(f"expect an iterable of futures, not
{type(fs).__name__}")
...
Because generators are coroutines, and the first condition in as_completed() is
True, and TypeError gets raised:
from asyncio import coroutines
# generators and generator expressions are coroutines
assert coroutines.iscoroutine(example() for _ in range(10))
Perhaps as_completed() can use inspect.isawaitable() instead, like so:
from inspect import isawaitable
def as_completed(fs, *, loop=None, timeout=None):
if futures.isfuture(fs) or isawaitable(fs):
...
I made a pull request with that change here[2].
[1] https://docs.python.org/3/library/asyncio-task.html#asyncio.as_completed
[2] https://github.com/python/cpython/pull/26228
--
components: asyncio
files: as_completed_gen.py
messages: 393923
nosy: alexdelorenzo, asvetlov, yselivanov
priority: normal
pull_requests: 24845
severity: normal
status: open
title: asyncio.as_completed() raises TypeError when the first supplied
parameter is a generator that yields awaitables
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file50052/as_completed_gen.py
___
Python tracker
<https://bugs.python.org/issue44176>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
