[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

Changes by Berker Peksag : -- components: +Library (Lib) superseder: -> Make Executor.map work with infinite/large inputs correctly ___ Python tracker ___ __

[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

Klamann added the comment: Thanks for pointing this out. *closed* -- resolution: -> duplicate stage: -> resolved status: open -> closed ___ Python tracker ___ _

[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

Josh Rosenberg added the comment: This is a strict duplicate of #29842, for which I've had a PR outstanding for a couple months now. -- nosy: +josh.r ___ Python tracker ___

[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

Klamann added the comment: Yes, I was wrong in my assumption that simply replacing the list comprehension with a generator expression would fix the issue. Nevertheless, there is no need to load the *entire* generator into memory by converting it to a list. All we have to read are the first n e

[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

Xiang Zhang added the comment: IIUC, the meaning of executor.map() is to run the tasks concurrently. So you have to loop the iterable ahead to submit all the tasks. If you make it a generator comprehension, you are actually submitting a task and then getting the result. So you are executing th

[issue30323] concurrent.futures.Executor.map() consumes all memory when big generators are used

New submission from Klamann: The Executor's map() function accepts a function and an iterable that holds the function arguments for each call to the function that should be made. This iterable could be a generator, and as such it could reference data that won't fit into memory. The behaviour