[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: This should have fixed it. If now, someone reopen the issue :) -- resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
STINNER Victor victor.stin...@gmail.com added the comment: This should have fixed it. If now, someone reopen the issue :) Thanks! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset f163c4731c58 by Antoine Pitrou in branch 'default': Issue #14666: stop multiprocessing's resource-sharing thread after the tests are done. http://hg.python.org/cpython/rev/f163c4731c58 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Richard Oudkerk shibt...@gmail.com added the comment: New patch which adds timeout to ResourceSharer.stop() which defaults to 0. When stop() fails it now uses the logger. pthread_sigmask() only stops this background thread from receiving signals. Signals will still be delivered to other threads, so it should not have any effect on the handling of Ctrl-C. -- Added file: http://bugs.python.org/file25371/mp_resource_sharer_stop.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Richard Oudkerk shibt...@gmail.com added the comment: New version of patch which does signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG)) in the thread (is that right?). It also uses a timeout when trying to join the thread. -- Added file: http://bugs.python.org/file25361/mp_resource_sharer_stop.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: in the thread (is that right?). This looks like it. It also uses a timeout when trying to join the thread. Perhaps some kind of warning can be printed if joining fails after the timeout? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Richard Oudkerk shibt...@gmail.com added the comment: Warning added to patch. -- Added file: http://bugs.python.org/file25362/mp_resource_sharer_stop.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: Hmm, I thought either multiprocessing's logging facilities, or the warnings module, could be used. That way, people have a control over verbosity of stderr messages. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
STINNER Victor victor.stin...@gmail.com added the comment: mp_resource_sharer_stop.patch: this patch changes two different things, the patch should be splitted. One patch to fix test_socket. One patch to call pthread_sigmask(). I don't think that you should call pthread_sigmask(). It looks like a workaround for this issue, whereas resource_sharer.stop() is the correct fix. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: I don't think that you should call pthread_sigmask(). It looks like a workaround for this issue, whereas resource_sharer.stop() is the correct fix. The problem is not only with test_multiprocessing and test_socket; any test which uses multiprocessing could have side effects on any subsequent tests which uses signals. Also, applicative code could be affected. So I think pthread_sigmask() *is* the solution. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
STINNER Victor victor.stin...@gmail.com added the comment: mp_resource_sharer_stop.patch: you should add a timeout argument to stop() instead of hardcoding a timeout of 5 seconds. It is maybe safer to block until the thread exits by default (so timeout=None by default). For the new method: it may be nice to document it. Having to import resource_sharer from multiprocessing.reduction is maybe not the best possible API :-/ +from multiprocessing.reduction import resource_sharer +resource_sharer.stop() Also, applicative code could be affected. What is the effect of the patch? For example, on CTRL+c? I don't know the multiprocessing module nor this resource sharer thread. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: For the new method: it may be nice to document it. Having to import resource_sharer from multiprocessing.reduction is maybe not the best possible API :-/ resource_sharer is a private API, it's not meant to be used by anyone outside of the stdlib. What is the effect of the patch? For example, on CTRL+c? Why should it have an effect on CTRL+c? Please explain yourself better. I don't know the multiprocessing module nor this resource sharer thread. Time to learn about them perhaps :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
New submission from STINNER Victor victor.stin...@gmail.com: [233/364] test_multiprocessing ... [265/364] test_typechecks [266/364] test_socket Timeout (1:00:00)! Thread 0x000807235000: File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/socket.py, line 135 in accept File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/multiprocessing/connection.py, line 595 in accept File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/multiprocessing/connection.py, line 469 in accept File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/multiprocessing/reduction.py, line 256 in _serve File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/threading.py, line 592 in run File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/threading.py, line 635 in _bootstrap_inner File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/threading.py, line 612 in _bootstrap Thread 0x000801407400: File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_socket.py, line 1208 in check_sendall_interrupted File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_socket.py, line 1219 in test_sendall_interrupted File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/case.py, line 385 in _executeTestPart File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/case.py, line 440 in run File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/case.py, line 492 in __call__ File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/suite.py, line 105 in run File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/suite.py, line 67 in __call__ File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/suite.py, line 105 in run File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/suite.py, line 67 in __call__ File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/unittest/runner.py, line 168 in run File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/support.py, line 1333 in _run_suite File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/support.py, line 1367 in run_unittest File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/test_socket.py, line 4813 in test_main File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/regrtest.py, line 1237 in runtest_inner File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/regrtest.py, line 907 in runtest File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/regrtest.py, line 710 in main File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/test/__main__.py, line 13 in module File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/runpy.py, line 73 in _run_code File /usr/home/buildbot/buildarea/3.x.krah-freebsd/build/Lib/runpy.py, line 160 in _run_module_as_main *** Error code 1 http://www.python.org/dev/buildbot/all/builders/AMD64%20FreeBSD%209.0%203.x/builds/2339/steps/test/logs/stdio -- messages: 159230 nosy: haypo, neologix, pitrou priority: normal severity: normal status: open title: test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread versions: Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
STINNER Victor victor.stin...@gmail.com added the comment: There was a similar issue: #11753, but it was a bug in the faulthandler module. Here it looks like a bug in TestSocketSharing of test_socket which uses multiprocessing. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: Ah, this is because of the new daemon thread in ResourceSharer. That thread is never stopped and could receive signals while tests expect them to be delivered to the main thread. Either we add a (private?) facility to stop that thread, or we block signal delivery in that thread using the signal module's pthread_sigmask. What do you think? -- nosy: +sbt ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Changes by Antoine Pitrou pit...@free.fr: -- components: +Library (Lib) stage: - needs patch type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Antoine Pitrou pit...@free.fr added the comment: The pthread_sigmask() solution would allow the use of multiprocessing all the while keeping deterministic signal delivery. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14666] test_sendall_interrupted hangs on FreeBSD with a zombi multiprocessing thread
Richard Oudkerk shibt...@gmail.com added the comment: This patch adds a ResourceSharer.stop() method. This is called from tearDownClass() in the unittest. -- keywords: +patch Added file: http://bugs.python.org/file25357/mp_resource_sharer_stop.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14666 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
