[issue30703] test_multiprocessing_forkserver hangs on the master branch

STINNER Victor added the comment: If I understood corretly, the problem is that the Python C signal handler is not reentrant because it calls Py_AddPendingCall() which uses a lock and a list. Before, the signal handler queued a new call to checksignals_witharg() (which just calls

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Changes by Antoine Pitrou : -- type: -> behavior versions: -Python 2.7, Python 3.5, Python 3.6 ___ Python tracker ___

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Changes by Antoine Pitrou : -- dependencies: +Make CHECK_STATUS_PTHREAD signal-safe, PyThread_acquire_lock can block even when asked not ot ___ Python tracker

[issue30703] test_multiprocessing_forkserver hangs on the master branch

STINNER Victor added the comment: > It is actually the same issue as https://bugs.python.org/issue11768 (which > was wrongly closed as fixed, apparently :-)) Well, don't be confused by the issue title. The only made change is to only call Py_AddPendingCall() only once, instead of calling it

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Antoine Pitrou added the comment: Using pthread_mutex_trylock() *and* disabling the CHECK_STATUS_PTHREAD() calls (which use the non-async-signal-safe fprintf()) at least seems to suppress the hangs on Matt's OS X machine (after more than 1000 runs). --

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Antoine Pitrou added the comment: Ok, I think I've managed to dig to the core issue. It is actually the same issue as https://bugs.python.org/issue11768 (which was wrongly closed as fixed, apparently :-)). Py_AddPendingCall() calls PyThread_acquire_lock() to try and take the pending calls

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Matt Billenstein added the comment: Yes, I'll email you the details. -- ___ Python tracker ___ ___

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Antoine Pitrou added the comment: Thanks for trying :-) Hmm, that's annoying. I don't know if you'd like to give me shell access to the machine (and a CPython checkout I can play with, perhaps)? -- ___ Python tracker

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Matt Billenstein added the comment: I don't see anything odd -- it runs for awhile and then times out once it's deadlocked: 0:03:18 load avg: 3.20 [224] test_multiprocessing_forkserver test_many_processes (test.test_multiprocessing_forkserver.WithProcessesTestProcess) ... ok

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Antoine Pitrou added the comment: It would be nice to know if there is additional output (for example exceptions happening in other threads or processes) when you run that command and manage to trigger a hang. -- ___ Python tracker

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Matt Billenstein added the comment: Cool -- do you need me to do something more to help debug this? -- ___ Python tracker ___

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Changes by Antoine Pitrou : -- title: regrtest hangs on the master branch -> test_multiprocessing_forkserver hangs on the master branch ___ Python tracker

[issue30703] test_multiprocessing_forkserver hangs on the master branch

Antoine Pitrou added the comment: Matt, if you try the following command, it will run the specific test in a loop in verbose mode: $ ./python -m test --timeout=30 -F -m test_many_processes -v test_multiprocessing_forkserver -- nosy: +pitrou ___