New submission from Ionel Cristian Mărieș <cont...@ionelmc.ro>:
Running `python3.8 mp-bug-python2.8.py` usually gets stuck after a dozen
iterations or so.
It appears that if I stop setting that signal handler it doesn't get stuck.
Unfortunately I need it to perform critical cleanup.
This is what I got from gdb:
(gdb) py-bt
Traceback (most recent call first):
File "/usr/lib/python3.8/multiprocessing/synchronize.py", line 95, in
__enter__
return self._semlock.__enter__()
File "/usr/lib/python3.8/multiprocessing/queues.py", line 355, in get
with self._rlock:
File "/usr/lib/python3.8/multiprocessing/pool.py", line 370, in worker
`func` and (a, b) becomes func(a, b).
File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.8/multiprocessing/process.py", line 569, in _bootstrap
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch
code = process_obj._bootstrap(parent_sentinel=child_r)
File "/usr/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/usr/lib/python3.8/multiprocessing/context.py", line 276, in _Popen
return Popen(process_obj)
File "/usr/lib/python3.8/multiprocessing/process.py", line 633, in start
File "/usr/lib/python3.8/multiprocessing/pool.py", line 838, in
_repopulate_pool_static
self._length = None
File "/usr/lib/python3.8/multiprocessing/pool.py", line 559, in
_repopulate_pool
outqueue.put(None)
File "/usr/lib/python3.8/multiprocessing/pool.py", line 212, in __init__
self._repopulate_pool()
File "/usr/lib/python3.8/multiprocessing/context.py", line 375, in Pool
File "mp-bug-python2.8.py", line 21, in <module>
And without the macros:
#0 0x00007f51c79fb6d6 in futex_abstimed_wait_cancelable (private=128,
abstime=0x0, expected=0, futex_word=0x7f51c7e29000) at
../sysdeps/unix/sysv/linux/futex-internal.h:205
#1 do_futex_wait (sem=sem@entry=0x7f51c7e29000, abstime=0x0) at
sem_waitcommon.c:111
#2 0x00007f51c79fb7c8 in __new_sem_wait_slow (sem=0x7f51c7e29000, abstime=0x0)
at sem_waitcommon.c:181
#3 0x00007f51c79fb839 in __new_sem_wait (sem=<optimized out>) at sem_wait.c:42
#4 0x00007f51c58d51f7 in semlock_acquire (self=0x7f51c58a60b0, args=<optimized
out>, kwds=<optimized out>) at ./Modules/_multiprocessing/semaphore.c:319
#5 0x000000000067d929 in method_vectorcall_VARARGS_KEYWORDS
(func=<method_descriptor at remote 0x7f51c6349c70>, args=0x7f51c58a4eb8,
nargsf=<optimized out>, kwnames=<optimized out>)
at ../Objects/descrobject.c:332
#6 0x000000000042c5dc in _PyObject_Vectorcall (kwnames=<optimized out>,
nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at
../Include/cpython/abstract.h:127
#7 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic
pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#8 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
../Python/ceval.c:3486
#9 0x0000000000425ed7 in function_code_fastcall (co=<optimized out>,
args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#10 0x00000000006761e4 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1,
args=0x7ffe20bcc908, callable=<function at remote 0x7f51c6356820>) at
../Include/cpython/abstract.h:127
#11 method_vectorcall (method=<optimized out>, args=<optimized out>,
nargsf=<optimized out>, kwnames=0x0) at ../Objects/classobject.c:67
#12 0x000000000042857e in _PyObject_Vectorcall (kwnames=0x0, nargsf=0,
args=0x0, callable=<method at remote 0x7f51c665efc0>) at
../Include/cpython/abstract.h:127
#13 _PyObject_CallNoArg (func=<optimized out>) at
../Include/cpython/abstract.h:153
#14 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
../Python/ceval.c:3287
#15 0x0000000000425ed7 in function_code_fastcall (co=<optimized out>,
args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#16 0x0000000000676245 in _PyObject_Vectorcall (kwnames=0x0, nargsf=1,
args=0x23cea18, callable=<function at remote 0x7f51c63563a0>) at
../Include/cpython/abstract.h:127
#17 method_vectorcall (method=<optimized out>, args=0x23cea20,
nargsf=<optimized out>, kwnames=0x0) at ../Objects/classobject.c:60
#18 0x000000000042cf2f in _PyObject_Vectorcall (kwnames=<optimized out>,
nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at
../Include/cpython/abstract.h:127
#19 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic
pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#20 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
../Python/ceval.c:3500
#21 0x00000000004f9f44 in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x23ce820, for file /usr/lib/python3.8/multiprocessing/pool.py,
line 370, in worker (inqueue=<SimpleQueue(_reader=<Connection(_handle=3,
_readable=True, _writable=False) at remote 0x7f51c589dcd0>,
_writer=<Connection(_handle=None, _readable=False, _writable=True) at remote
0x7f51c589d940>, _rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote
0x7f51c58a60b0>, acquire=<built-in method acquire of _multiprocessing.SemLock
object at remote 0x7f51c58a60b0>, release=<built-in method release of
_multiprocessing.SemLock object at remote 0x7f51c58a60b0>) at remote
0x7f51c589d0d0>, _poll=<method at remote 0x7f51c58ae2c0>,
_wlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a1030>,
acquire=<built-in method acquire of _multiprocessing.SemLock object at remote
0x7f51c58a1030>, release=<built-in method release of _multiprocessing.SemLock
object at remote 0x7f51c58a1030>) at remote 0x7f51c589deb0>) at remote
0x7f51c589d3a0>, outqueue=<SimpleQueue(_reader=<Connection(_handle=None,
_readable=True, _wr...(truncated))
at ../Python/ceval.c:741
#22 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>,
locals=locals@entry=0x0, args=<optimized out>, argcount=6, kwnames=0x0,
kwargs=0x7f51c636e628, kwcount=0, kwstep=1,
defs=0x7f51c65a4828, defcount=4, kwdefs=0x0, closure=0x0, name='worker',
qualname='worker') at ../Python/ceval.c:4298
#23 0x000000000043e7a2 in _PyFunction_Vectorcall (func=func@entry=<function at
remote 0x7f51c6352d30>, stack=<optimized out>, nargsf=nargsf@entry=6,
kwnames=<optimized out>) at ../Objects/call.c:435
#24 0x000000000044143c in PyVectorcall_Call (callable=<function at remote
0x7f51c6352d30>, tuple=<optimized out>, kwargs=<optimized out>) at
../Objects/call.c:199
#25 0x0000000000428aec in do_call_core (kwdict={},
callargs=(<SimpleQueue(_reader=<Connection(_handle=3, _readable=True,
_writable=False) at remote 0x7f51c589dcd0>, _writer=<Connection(_handle=None,
_readable=False, _writable=True) at remote 0x7f51c589d940>,
_rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a60b0>,
acquire=<built-in method acquire of _multiprocessing.SemLock object at remote
0x7f51c58a60b0>, release=<built-in method release of _multiprocessing.SemLock
object at remote 0x7f51c58a60b0>) at remote 0x7f51c589d0d0>, _poll=<method at
remote 0x7f51c58ae2c0>, _wlock=<Lock(_semlock=<_multiprocessing.SemLock at
remote 0x7f51c58a1030>, acquire=<built-in method acquire of
_multiprocessing.SemLock object at remote 0x7f51c58a1030>, release=<built-in
method release of _multiprocessing.SemLock object at remote 0x7f51c58a1030>) at
remote 0x7f51c589deb0>) at remote 0x7f51c589d3a0>,
<SimpleQueue(_reader=<Connection(_handle=None, _readable=True, _writable=False)
at remote 0x7f51c58a7670>, _writer=<Connection(_handle=6, _readable=False,
_writable=True) at...(truncated),
func=<function at remote 0x7f51c6352d30>, tstate=0x22bd160) at
../Python/ceval.c:5034
#26 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
../Python/ceval.c:3559
#27 0x0000000000425ed7 in function_code_fastcall (co=<optimized out>,
args=<optimized out>, nargs=1, globals=<optimized out>) at ../Objects/call.c:283
#28 0x000000000042c5dc in _PyObject_Vectorcall (kwnames=<optimized out>,
nargsf=<optimized out>, args=<optimized out>, callable=<optimized out>) at
../Include/cpython/abstract.h:127
#29 call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic
pointer>, tstate=0x22bd160) at ../Python/ceval.c:4987
#30 _PyEval_EvalFrameDefault (f=<optimized out>, throwflag=<optimized out>) at
../Python/ceval.c:3486
#31 0x00000000004f9f44 in PyEval_EvalFrameEx (throwflag=0,
f=Frame 0x23e7f40, for file /usr/lib/python3.8/multiprocessing/process.py,
line 569, in _bootstrap (self=<ForkProcess(_identity=(98,), _config={'authkey':
<AuthenticationString at remote 0x7f51c66754c0>, 'semprefix': '/mp', 'daemon':
True}, _parent_pid=191540, _parent_name='MainProcess', _popen=None,
_closed=False, _target=<function at remote 0x7f51c6352d30>,
_args=(<SimpleQueue(_reader=<Connection(_handle=3, _readable=True,
_writable=False) at remote 0x7f51c589dcd0>, _writer=<Connection(_handle=None,
_readable=False, _writable=True) at remote 0x7f51c589d940>,
_rlock=<Lock(_semlock=<_multiprocessing.SemLock at remote 0x7f51c58a60b0>,
acquire=<built-in method acquire of _multiprocessing.SemLock object at remote
0x7f51c58a60b0>, release=<built-in method release of _multiprocessing.SemLock
object at remote 0x7f51c58a60b0>) at remote 0x7f51c589d0d0>, _poll=<method at
remote 0x7f51c58ae2c0>, _wlock=<Lock(_semlock=<_multiprocessing.SemLock at
remote 0x7f51c58a1030>, acquire=<built-in method acquire of
_multiprocessin...(truncated))
at ../Python/ceval.c:741
#32 _PyEval_EvalCodeWithName (_co=<optimized out>, globals=<optimized out>,
locals=locals@entry=0x0, args=<optimized out>, argcount=1,
kwnames=0x7f51c589d5c8, kwargs=0x7f51c66fcde8, kwcount=1, kwstep=1,
defs=0x7f51c6672808, defcount=1, kwdefs=0x0, closure=0x0,
name='_bootstrap', qualname='BaseProcess._bootstrap') at ../Python/ceval.c:4298
#33 0x000000000043e7a2 in _PyFunction_Vectorcall (func=<optimized out>,
stack=<optimized out>, nargsf=<optimized out>, kwnames=<optimized out>) at
../Objects/call.c:435
#34 0x0000000000676245 in _PyObject_Vectorcall (kwnames=('parent_sentinel',),
nargsf=1, args=0x7f51c66fcde0, callable=<function at remote 0x7f51c6681160>) at
../Include/cpython/abstract.h:127
#35 method_vectorcall (method=<optimized out>, args=0x7f51c66fcde8,
nargsf=<optimized out>, kwnames=('parent_sentinel',)) at
../Objects/classobject.c:60
This issue doesn't reproduce on python 3.7 or earlier.
----------
files: mp-bug-python2.8.py
messages: 352815
nosy: ionelmc
priority: normal
severity: normal
status: open
title: Setting a signal handler gets multiprocessing.Pool stuck
versions: Python 3.8
Added file: https://bugs.python.org/file48615/mp-bug-python2.8.py
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue38227>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
