[issue41505] asyncio.gather of large streams with limited resources

2020-08-23 Thread Kevin Amado
Kevin Amado added the comment: Yeah definitely it must be workers I've experimented a lot about it and finally found something with an interface similar to asyncio.as_completed - You control concurrency with `workers` parameter - You upper-bound memory usage with `worker_greediness

[issue41505] asyncio.gather of large streams with limited resources

2020-08-07 Thread Kevin Amado
Change by Kevin Amado : Removed file: https://bugs.python.org/file49377/materialize-implementation.py ___ Python tracker <https://bugs.python.org/issue41505> ___ ___

[issue41505] asyncio.gather of large streams with limited resources

2020-08-07 Thread Kevin Amado
New submission from Kevin Amado : Sometimes when dealing with high concurrency systems developers face the problem of executing concurrently a large number of tasks while taking care of a finite pool of resources Just to mention some examples: - reading asynchronously a lot of files without