[issue41699] Potential memory leak with asyncio and run_in_executor

Kyle Stanley added the comment: > Regularly calling executor.shutdown() and then instantiating a new > ThreadPoolExecutor in order to run an asyncio program does not seem like a > good API to me. Clarification: you're typically only supposed to instantiate a single ThreadPoolExecutor or Pro

[issue41699] Potential memory leak with asyncio and run_in_executor

Sophia Wisdom added the comment: While not calling executor.shutdown() may leave some resources still used, it should be small and fixed. Regularly calling executor.shutdown() and then instantiating a new ThreadPoolExecutor in order to run an asyncio program does not seem like a good API to

[issue41699] Potential memory leak with asyncio and run_in_executor

Kyle Stanley added the comment: Also note that the difference in memory is much higher when an exception occurs (presumably because the exception is stored on future._exception and not cleaned up?): ``` [aeros:~/repos/cpython]$ ./python ~/programming/python/asyncio_run_in_exec_leak.py -n=100

[issue41699] Potential memory leak with asyncio and run_in_executor

Kyle Stanley added the comment: In the snippet provided, at least part of the resources are not finalized because executor.shutdown() was not called in the program (which should be done when creating a local instance of the executors, either explicitly or using the context manager). For the

[issue41699] Potential memory leak with asyncio and run_in_executor

Sophia Wisdom added the comment: It looks like it's not specific to the ThreadPoolExecutor. ``` import asyncio import concurrent def leaker_func(): list(range(int(1000))) # removed 1/0 because this causes issues with the ProcessPoolExecutor async def function(): loop = asyncio.ge

[issue41699] Potential memory leak with asyncio and run_in_executor

Daniel Alley added the comment: This seems likely to be a duplicate of https://bugs.python.org/issue41588, as run_in_executor(None, ...) submits tasks to a ThreadPoolExecutor underneath the hood. -- nosy: +dralley ___ Python tracker

[issue41699] Potential memory leak with asyncio and run_in_executor

Change by Karthikeyan Singaravelan : -- nosy: +aeros ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue41699] Potential memory leak with asyncio and run_in_executor

New submission from Sophia Wisdom : The below example leaks ~20 megabytes of memory. The amount leaked is related to both the number of items in the list and the number of times `run_in_executor` is called. ``` import asyncio def leaker(): x = list(range(int(1000))) 1/0 async def fu