New issue 3060: Multiprocessing regression in PyPy 7.x https://bitbucket.org/pypy/pypy/issues/3060/multiprocessing-regression-in-pypy-7x
Nathan Stocks: I'm the maintainer of [Green](https://github.com/CleanCut/green) and I've run across what seems to be a very odd pypy3 bug in `multiprocessing/connection.py` specific to macOS. This doesn't happen on any other version of Python nor on any platform that I test, also this didn’t happen on PyPy 6.0 and earlier, so I don’t suspect the python module itself, per se. OS: macOS Mojave 10.14.6 Pypy: `Python 3.6.1 (784b254d669919c872a505b807db8462b6140973, Jun 18 2019, 05:54:52)` / `[PyPy 7.1.1-beta0 with GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.46.4)] on darwin` Steps to reproduce: 1. Install green * `pip_pypy3 install green` 1. Run a portion of green's self-tests: * `/usr/local/share/pypy3/green -vvv green.test.test_runner` I observe 3\(!!!\) different crashes, each of which seem to implicate `multiprocessing/connection.py`. As far as I can tell, which error you get is simply random chance, though the first type seems to occur most often for me. Crash Type 1 seems to be the most common, an indefinite hang which requires pressing Ctrl-C to break out of: ``` $ /usr/local/share/pypy3/green -vvv green.test.test_runner Green 2.16.1, Coverage 4.5.4, Python 3.6.1 green.test.test_runner TestInitializerOrFinalizer . Given a blank dotted function, calling the initializer/finalizer does nothing. . An importable, callable object...crashes. . Given an actual importable module and function, the function is run. . An importable, but not-callable-object also raises an InitializerOrFinalizerError. . Given an unimportable module, an InitializerOrFinalizerError is raised. TestProcesses . Bad syntax in a testfile is caught as a test error. . run() can catch SIGINT while running a process. . If tempfile.gettempdir() is used for dir, using same testfile name will not collide. . args.processes = 0 causes auto-detection of number of processes. . run() does not crash with empty suite and processes . Running coverage in process mode doesn't crash . Exceptions that escape the test framework get caught by poolRunner and reported as a failure. For example, the testtools implementation of TestCase unwisely (but deliberately) lets SystemExit exceptions through. TestRun . run() can use a GreenStream for output. . A failing test case causes the whole run to report 'FAILED' . failfast causes the testing to stop after the first failure. . When we don't find any tests, we say so. . run() can use sys.stdout as the stream. . Raising a SystemExit gets caught and reported. . verbose=3 causes version output, and an empty test case passes. E setting warnings='always' doesn't crash ^CProcess DaemonlessPoolWorker-1: Traceback (most recent call last): File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/process.py", line 252, in _bootstrap File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/util.py", line 288, in _exit_function File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/util.py", line 254, in _run_finalizers File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/util.py", line 186, in __call__ File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/managers.py", line 615, in _finalize_manager process.join(timeout=1.0) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/process.py", line 121, in join File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/popen_fork.py", line 48, in wait File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 911, in wait ready = selector.select(timeout) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/selectors.py", line 376, in select File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/signals.py", line 36, in __call__ KeyboardInterrupt Error in green.test.test_runner.TestRun.test_warnings File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 59, in testPartExecutor yield File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 601, in run testMethod() File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/test/test_runner.py", line 182, in test_warnings result = run(tests, self.stream, self.args) File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/runner.py", line 87, in run manager = multiprocessing.Manager() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/context.py", line 56, in Manager m.start() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/managers.py", line 503, in start reader, writer = connection.Pipe(duplex=False) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 512, in Pipe fd1, fd2 = os.pipe() OSError: [Errno 24] Too many open files Warning: Some tests may not have been run. Ran 20 tests in 10.948s using 8 processes FAILED (errors=1, passes=19) ``` Here is Crash Type 2 talking about "bad message length": ``` $ /usr/local/share/pypy3/green -vvv green.test.test_runner Green 2.16.1, Coverage 4.5.4, Python 3.6.1 green.test.test_runner TestInitializerOrFinalizer . Given a blank dotted function, calling the initializer/finalizer does nothing. . An importable, callable object...crashes. . Given an actual importable module and function, the function is run. . An importable, but not-callable-object also raises an InitializerOrFinalizerError. . Given an unimportable module, an InitializerOrFinalizerError is raised. TestProcesses . Bad syntax in a testfile is caught as a test error. . run() can catch SIGINT while running a process. . If tempfile.gettempdir() is used for dir, using same testfile name will not collide. . args.processes = 0 causes auto-detection of number of processes. . run() does not crash with empty suite and processes . Running coverage in process mode doesn't crash . Exceptions that escape the test framework get caught by poolRunner and reported as a failure. For example, the testtools implementation of TestCase unwisely (but deliberately) lets SystemExit exceptions through. TestRun . run() can use a GreenStream for output. . A failing test case causes the whole run to report 'FAILED' . failfast causes the testing to stop after the first failure. . When we don't find any tests, we say so. . run() can use sys.stdout as the stream. . Raising a SystemExit gets caught and reported. . verbose=3 causes version output, and an empty test case passes. E setting warnings='always' doesn't crash Error in green.test.test_runner.TestRun.test_warnings File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 59, in testPartExecutor yield File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 601, in run testMethod() File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/test/test_runner.py", line 182, in test_warnings result = run(tests, self.stream, self.args) File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/runner.py", line 89, in run for target in toParallelTargets(suite, args.targets)] File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/runner.py", line 89, in <listcomp> for target in toParallelTargets(suite, args.targets)] File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/managers.py", line 662, in temp token, exp = self._create(typeid, *args, **kwds) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/managers.py", line 554, in _create conn = self._Client(self._address, authkey=self._authkey) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 493, in Client answer_challenge(c, authkey) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 732, in answer_challenge message = connection.recv_bytes(256) # reject large message File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 218, in recv_bytes self._bad_message_length() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 151, in _bad_message_length raise OSError("bad message length") OSError: bad message length Ran 20 tests in 5.613s using 8 processes FAILED (errors=1, passes=19) ``` And finally Crash Type 3, talking about an "EOF Error": ``` $ /usr/local/share/pypy3/green -vvv green.test.test_runner Green 2.16.1, Coverage 4.5.4, Python 3.6.1 green.test.test_runner TestInitializerOrFinalizer . Given a blank dotted function, calling the initializer/finalizer does nothing. . An importable, callable object...crashes. . Given an actual importable module and function, the function is run. . An importable, but not-callable-object also raises an InitializerOrFinalizerError. . Given an unimportable module, an InitializerOrFinalizerError is raised. TestProcesses . Bad syntax in a testfile is caught as a test error. . run() can catch SIGINT while running a process. . If tempfile.gettempdir() is used for dir, using same testfile name will not collide. . args.processes = 0 causes auto-detection of number of processes. . run() does not crash with empty suite and processes . Running coverage in process mode doesn't crash . Exceptions that escape the test framework get caught by poolRunner and reported as a failure. For example, the testtools implementation of TestCase unwisely (but deliberately) lets SystemExit exceptions through. TestRun . run() can use a GreenStream for output. . A failing test case causes the whole run to report 'FAILED' . failfast causes the testing to stop after the first failure. . When we don't find any tests, we say so. . run() can use sys.stdout as the stream. . Raising a SystemExit gets caught and reported. . verbose=3 causes version output, and an empty test case passes. E setting warnings='always' doesn't crash Error in green.test.test_runner.TestRun.test_warnings File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 59, in testPartExecutor yield File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/unittest/case.py", line 601, in run testMethod() File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/test/test_runner.py", line 182, in test_warnings result = run(tests, self.stream, self.args) File "/usr/local/Cellar/pypy3/7.1.1/libexec/site-packages/green/runner.py", line 87, in run manager = multiprocessing.Manager() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/context.py", line 56, in Manager m.start() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/managers.py", line 517, in start self._address = reader.recv() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 407, in _recv_bytes buf = self._recv(4) File "/usr/local/Cellar/pypy3/7.1.1/libexec/lib-python/3/multiprocessing/connection.py", line 383, in _recv raise EOFError EOFError Ran 20 tests in 4.402s using 8 processes FAILED (errors=1, passes=19) ``` _______________________________________________ pypy-issue mailing list pypy-issue@python.org https://mail.python.org/mailman/listinfo/pypy-issue
