[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

miss-islington added the comment: New changeset 216c040bb1fdf73e78d67ab82a43563d7593f874 by Miss Islington (bot) in branch '3.9': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/216c040bb1fdf73e78d67ab82a43563d7593f874 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by Gregory P. Smith : -- resolution: -> fixed stage: patch review -> commit review status: open -> closed ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

miss-islington added the comment: New changeset 8f6aa48cb2dc827a2cb76e35e91bf02d099875c5 by Miss Islington (bot) in branch '3.10': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/8f6aa48cb2dc827a2cb76e35e91bf02d099875c5 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by miss-islington : -- message_count: 10.0 -> 11.0 nosy: +miss-islington nosy_count: 6.0 -> 7.0 pull_requests: +27420 pull_request: https://github.com/python/cpython/pull/29144 ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Gregory P. Smith added the comment: New changeset 843b890334ca30cf6af27dffe29cecd06b49f7d9 by Gregory P. Smith in branch 'main': bpo-43592: Raise RLIMIT_NOFILE in test.libregrtest (GH-29127) https://github.com/python/cpython/commit/843b890334ca30cf6af27dffe29cecd06b49f7d9 --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by miss-islington : -- pull_requests: +27421 pull_request: https://github.com/python/cpython/pull/29145 ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Change by Gregory P. Smith : -- keywords: +patch pull_requests: +27414 pull_request: https://github.com/python/cpython/pull/29127 ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Gregory P. Smith added the comment: macOS often has an unreasonably low default max number of file descriptors. This is easy to raise from user code, it's a soft limit. https://github.com/python/cpython/pull/29127 is out to try that from regrtest itself. -- assignee: ->

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: FYI the test is still failing with the same error on the billenstein-macos builder worker, x86-64 macOS 3.10: https://buildbot.python.org/all/#/builders/681/builds/370 "OSError: [Errno 24] Too many open files" --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Łukasz Langa added the comment: Before you do it, check the defaults with: $ launchctl limit maxfiles I'm curious. -- ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Łukasz Langa added the comment: Try setting the following: $ sudo sysctl -w kern.maxfiles=65536 $ sudo sysctl -w kern.maxfilesperproc=65536 $ sudo ulimit -n 65536 -- nosy: +lukasz.langa ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: > x86-64 macOS 3.10: https://buildbot.python.org/all/#/builders/681/builds/345 On this machine, RLIMIT_NOFILE soft limit is 256. test.pythoninfo: resource.RLIMIT_NOFILE: (256, 9223372036854775807) platform.platform: macOS-11.5.2-x86_64-i386-64bit

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: x86-64 macOS 3.10: https://buildbot.python.org/all/#/builders/681/builds/345 == FAIL: test_multiprocessing_pool_circular_import (test.test_importlib.test_threaded_import.ThreadedImportTests)

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Liel Fridman added the comment: I have the same error when running the tests on my machine (Same macOS version). -- nosy: +lielfr ___ Python tracker ___

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

Ned Deily added the comment: FWIW, I don't recall ever seeing a "too many open files" failure for test_importlib on any of my macOS systems. -- nosy: +ned.deily ___ Python tracker

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

STINNER Victor added the comment: test_importlib only fails *randomly* on x86-64 macOS 3.x. For example, the build 970 is a success (but orange): "Tests result: FAILURE then SUCCESS". test_importlib failed with too many open files, and then passed. --

[issue43592] test_importlib: test_multiprocessing_pool_circular_import() fails with "Too many open files" error on os.pipe()

New submission from STINNER Victor : x86-64 macOS 3.x: https://buildbot.python.org/all/#/builders/366/builds/969 Build triggered by the commit 88d9983b561cd59e5f186d98227de0c1a022b498 which changes PyImport_Import(). The buildbot is running on macOS 10.15.7 (Darwin Kernel Version 19.6.0)