New submission from Skip Montanaro <[email protected]>:
I find that test_multiprocessing_spawn frequently hangs. Hitting Ctl-C then
rerunning "make test" generally works. Still, this behavior makes it
problematic to run testing unattended. I don't think I have an unusual
environment (XUbuntu 20.04, GCC 9.3.0). Here's some output from the currently
running/hanging process, just after the last other unit test completed.
0:03:42 load avg: 4.96 [421/422/21] test_concurrent_futures passed (2 min 57
sec) -- running: test_multiprocessing_spawn (2 min 59 sec)
0:04:12 load avg: 3.19 running: test_multiprocessing_spawn (3 min 29 sec)
0:04:42 load avg: 1.93 running: test_multiprocessing_spawn (3 min 59 sec)
0:05:12 load avg: 1.23 running: test_multiprocessing_spawn (4 min 29 sec)
0:05:42 load avg: 0.74 running: test_multiprocessing_spawn (4 min 59 sec)
...
0:22:12 load avg: 0.42 running: test_multiprocessing_spawn (21 min 29 sec)
0:22:42 load avg: 0.27 running: test_multiprocessing_spawn (21 min 59 sec)
0:23:12 load avg: 0.37 running: test_multiprocessing_spawn (22 min 29 sec)
I'm currerntly mostly messing around with the nogil code and 3.9 tip, but I've
noticed this problem on other branches as well. The fact that this seems not to
have been reported before suggests that I'm somehow an outlier. If it was
common, my guess is that the buildbots would experience this problem on a
regular basis.
Here is the current python process info (output of pgrep -fla python).
1572195 /home/skip/src/python/cpython/python -u -W default -bb -E -m test -r -w
-j 0 -u all,-largefile,-audio,-gui
1575193 /home/skip/src/python/cpython/python -bb -E -Wdefault -u -m
test.regrtest --worker-args [{"testdir": null, "verbose": 0, "quiet": false,
"exclude": false, "single": false, "randomize": true, "fromfile": null,
"findleaks": 1, "use_resources": ["curses", "network", "decimal", "cpu",
"subprocess", "urlfetch"], "trace": false, "coverdir": "coverage", "runleaks":
false, "huntrleaks": false, "verbose2": true, "print_slow": false,
"random_seed": 2495022, "use_mp": 10, "verbose3": false, "forever": false,
"header": false, "failfast": false, "match_tests": null, "ignore_tests": null,
"pgo": false, "timeout": null, "wait": false, "worker_args": null, "start":
null, "match_filename": null, "ignore_filename": null, "use": [["all",
"-largefile", "-audio", "-gui"]], "memlimit": null, "threshold": null,
"nowindows": false, "list_tests": false, "list_cases": false, "pgo_extended":
false, "fail_env_changed": false, "xmlpath": null, "tempdir":
"/home/skip/src/python/cpython/build/test_python_1
572195", "cleanup": false, "args": []}, "test_multiprocessing_spawn"]
1575489 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from
multiprocessing.resource_tracker import main;main(3)
1575547 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from
multiprocessing.forkserver import main; main(6, 7, ['__main__',
'test.test_multiprocessing_forkserver'], **{'sys_path':
['/home/skip/src/python/cpython',
'/home/skip/tmp/nogilpgo_build/lib/python39.zip',
'/home/skip/src/python/cpython/Lib',
'/home/skip/src/python/cpython/build/lib.linux-x86_64-3.9',
'/home/skip/.local/lib/python3.9/site-packages',
'/home/skip/tmp/nogilpgo_build/lib/python3.9/site-packages']})
1589033 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from
multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=4,
pipe_handle=12) --multiprocessing-fork
1589034 /home/skip/src/python/cpython/python -bb -E -Wdefault -c from
multiprocessing.spawn import spawn_main; spawn_main(tracker_fd=4,
pipe_handle=14) --multiprocessing-fork
At the moment I can't pstack either of the last two processes. Unfortunately, I
think I need to tweak ptrace.conf and reboot.
----------
messages: 405718
nosy: skip.montanaro
priority: normal
severity: normal
status: open
title: test_multiprocessing_spawn hangs sometimes
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue45714>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
