[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by Andrew Svetlov : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

miss-islington added the comment: New changeset 3b9d886567c4fc6279c2198b6711f0590dbf3336 by Miss Islington (bot) in branch '3.9': bpo-43498: Fix dictionary iteration error in _ExecutorManagerThread (GH-24868) https://github.com/python/cpython/commit/3b9d886567c4fc6279c2198b6711f0590dbf3336

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

miss-islington added the comment: New changeset 4b11d7118561a12322d3cfa76c5941690b241149 by Miss Islington (bot) in branch '3.10': bpo-43498: Fix dictionary iteration error in _ExecutorManagerThread (GH-24868) https://github.com/python/cpython/commit/4b11d7118561a12322d3cfa76c5941690b241149

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by miss-islington : -- pull_requests: +28069 pull_request: https://github.com/python/cpython/pull/29837 ___ Python tracker ___

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Andrew Svetlov added the comment: New changeset 7431448b817d3bf87f71661cf8f3d537807ab2e2 by Jakub Kulík in branch 'main': bpo-43498: Fix dictionary iteration error in _ExecutorManagerThread (GH-24868) https://github.com/python/cpython/commit/7431448b817d3bf87f71661cf8f3d537807ab2e2

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by miss-islington : -- nosy: +miss-islington nosy_count: 11.0 -> 12.0 pull_requests: +28068 pull_request: https://github.com/python/cpython/pull/29836 ___ Python tracker

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by Alex Waygood : -- nosy: +colesbury ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by Andrew Svetlov : -- assignee: -> asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Andrew Svetlov added the comment: Thanks for the report. Atomic copy (`list(self.processes.values()`) should fix the bug, sure. I doubt if writing a reliable test for this situation is possible; multithreading is hard. I think we can accept a patch without a test but with an inline comment

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Mark Ao added the comment: I'm experiencing the same issue on Python 3.10.0 when I execute the code that uses concurrent.futures.ProcessPoolExecutor. Exception in thread Thread-1: Traceback (most recent call last): File "/usr/local/lib/python3.10/threading.py", line 1009, in

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Jakub Kulik added the comment: I think that even if copy() doesn't fix it entirely, it's still much better than nothing. I never encountered the issue mentioned in bpo-40327, but I saw this issue several times a week (before applying the proposed patch). --

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Dennis Sweeney added the comment: It was mentioned in bpo-40327 that although copy() makes the situation much better, it doesn't solve the problem entirely, since the memory allocation of the copy() call can release the GIL. I don't know enough to know whether it would be worth it to add

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Thomas Petazzoni added the comment: For the record: we're seeing this issue ~50 times a day on our build infrastructure. -- ___ Python tracker ___

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Thomas Petazzoni added the comment: I can confirm we are seeing the same issue when building Python 3.9 in the context of Buildroot. See http://autobuild.buildroot.net/results/ae6/ae6c4ab292589a4e4442dfb0a1286349a9bf4d29/build-end.log for an example build result. This happens since we have

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Matt Whitlock added the comment: Observed this same failure mode on a Raspberry Pi 1 while running 'make install' on Python 3.9.5 with 9 concurrent workers. Exception in thread Thread-1: Traceback (most recent call last): File

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Jakub Kulik added the comment: I investigated a little bit more and found out that this happens when `ProcessPoolExecutor::_adjust_process_count()` adds a new process during the iteration. With the following change, I can reproduce this reliably every time: ---

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Kartik Subbarao added the comment: I'm seeing the same error with Python 3.9.2 on Fedora 33, with a script that uses ProcessPoolExecutor. -- nosy: +kartiksubbarao ___ Python tracker

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by Karthikeyan Singaravelan : -- nosy: +bquinlan, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

Change by Jakub Kulik : -- keywords: +patch pull_requests: +23630 stage: -> patch review pull_request: https://github.com/python/cpython/pull/24868 ___ Python tracker ___

[issue43498] "dictionary changed size during iteration" error in _ExecutorManagerThread

New submission from Jakub Kulik : Recently several of our Python 3.9 builds froze during `make install` with the following trace in logs: Listing .../components/python/python39/build/prototype/sparc/usr/lib/python3.9/lib2to3/tests/data/fixers/myfixes... Exception in thread Thread-1: