New submission from Thomas Mortensson: Unsure how to re-produce. Have a CoreDump file with the below backtrace. Our module is socc, we were running a Twisted application and Python core dumped giving us the following backtrace.
Looks like we failed to spawn a process wsing os.fork() due to a double free in PyThread_ReInitTLS. System Details: CentOS 6.5 Kernel - 2.6.32-696.el6.x86_64 Python 2.7.13 (default, Feb 6 2017, 15:27:35) [GCC 4.8.3 20140911 (Red Hat 4.8.3-10)] on linux2 GDB - GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el6 File "/usr/local/lib/python2.7/site-packages/socc/cloudTransport/transport.py", line 128, in executeProcessInThread line +128 ret = reactor.spawnProcess(execProtocol, args[0], args, env=os.environ, childFDs={0: "w", 1: "r", 2: "r"}) Python backtrace: (gdb) py-bt Traceback (most recent call first): File "/usr/local/lib/python2.7/site-packages/twisted/internet/process.py", line 382, in _fork self.pid = os.fork() File "/usr/local/lib/python2.7/site-packages/twisted/internet/process.py", line 677, in __init__ self._fork(path, uid, gid, executable, args, environment, fdmap=fdmap) File "/usr/local/lib/python2.7/site-packages/twisted/internet/posixbase.py", line 345, in spawnProcess processProtocol, uid, gid, childFDs) File "/usr/local/lib/python2.7/site-packages/socc/cloudTransport/transport.py", line 128, in executeProcessInThread File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/usr/local/lib/python2.7/site-packages/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/usr/local/lib/python2.7/site-packages/twisted/python/threadpool.py", line 196, in _worker result = context.call(ctx, function, *args, **kwargs) File "/usr/local/lib/python2.7/threading.py", line 754, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/usr/local/lib/python2.7/threading.py", line 774, in __bootstrap self.__bootstrap_inner() /* Forget everything not associated with the current thread id. * This function is called from PyOS_AfterFork(). It is necessary * because other thread ids which were in use at the time of the fork * may be reused for new threads created in the forked process. */ void PyThread_ReInitTLS(void) { long id = PyThread_get_thread_ident(); struct key *p, **q; if (!keymutex) return; /* As with interpreter_lock in PyEval_ReInitThreads() we just create a new lock without freeing the old one */ keymutex = PyThread_allocate_lock(); /* Delete all keys which do not match the current thread id */ q = &keyhead; while ((p = *q) != NULL) { if (p->id != id) { *q = p->next; free((void *)p); <-------------------------FAILED HERE /* NB This does *not* free p->value! */ } else q = &p->next; } } Installed python27-Logbook-debuginfo-0.9.1-1.el6_7.x86_64.rpm python27-PyYAML-debuginfo-3.12-1.x86_64.rpm python27-Twisted-debuginfo-14.0.0-1.x86_64.rpm python27-cffi-debuginfo-1.9.1-1.el6_7.x86_64.rpm python27-cryptography-debuginfo-1.7.2-1.el6_7.x86_64.rpm python27-debuginfo-2.7.13-2.el6_7.x86_64.rpm python27-psutil-debuginfo-5.1.3-1.x86_64.rpm python27-simplejson-debuginfo-3.10.0-1.el6_7.el6_7.x86_64.rpm python27-zope-interface-debuginfo-4.3.3-1.x86_64.rpm Backtrace is: #0 0x00007ffff6db0625 in raise () from /lib64/libc.so.6 #1 0x00007ffff6db1e05 in abort () from /lib64/libc.so.6 #2 0x00007ffff6dee537 in __libc_message () from /lib64/libc.so.6 #3 0x00007ffff6df3f4e in malloc_printerr () from /lib64/libc.so.6 #4 0x00007ffff6df6cad in _int_free () from /lib64/libc.so.6 #5 0x00007ffff7b082a9 in PyThread_ReInitTLS () at Python/thread.c:413 #6 0x00007ffff7b0fbb4 in PyOS_AfterFork () at Modules/signalmodule.c:1004 #7 0x00007ffff7b12cf6 in posix_fork (self=<optimized out>, noargs=<optimized out>) at Modules/posixmodule.c:3875 #8 0x00007ffff7ac8460 in call_function (oparg=<optimized out>, pp_stack=0x7fffdc043fb0) at Python/ceval.c:4336 #9 PyEval_EvalFrameEx (f=f@entry=0x36bb930, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #10 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=argcount@entry=7, kws=kws@entry=0x36b54d0, kwcount=1, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:3584 #11 0x00007ffff7ac8800 in fast_function (nk=<optimized out>, na=7, n=<optimized out>, pp_stack=0x7fffdc0441f0, func=<optimized out>) at Python/ceval.c:4447 #12 call_function (oparg=<optimized out>, pp_stack=0x7fffdc0441f0) at Python/ceval.c:4372 #13 PyEval_EvalFrameEx (f=f@entry=0x36b5270, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #14 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffdeaa5e38, argcount=10, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x7fffeb543068, defcount=defcount@entry=3, closure=0x0) at Python/ceval.c:3584 #15 0x00007ffff7a40760 in function_call (func=0x7fffeb429398, arg=0x7fffdeaa5e20, kw=0x0) at Objects/funcobject.c:523 #16 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffeb429398, arg=arg@entry=0x7fffdeaa5e20, kw=kw@entry=0x0) at Objects/abstract.c:2547 #17 0x00007ffff7a1cec5 in instancemethod_call (func=0x7fffeb429398, arg=0x7fffdeaa5e20, kw=0x0) at Objects/classobject.c:2602 #18 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffdc068820, arg=arg@entry=0x7fffea19c2d0, kw=kw@entry=0x0) at Objects/abstract.c:2547 #19 0x00007ffff7a7c5bf in slot_tp_init (self=0x7fffdc059710, args=0x7fffea19c2d0, kwds=0x0) at Objects/typeobject.c:5806 #20 0x00007ffff7a78ecf in type_call (type=<optimized out>, args=0x7fffea19c2d0, kwds=0x0) at Objects/typeobject.c:765 #21 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0xb89410, arg=arg@entry=0x7fffea19c2d0, kw=kw@entry=0x0) at Objects/abstract.c:2547 #22 0x00007ffff7ac452e in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffdc044820, func=<optimized out>) at Python/ceval.c:4569 #23 call_function (oparg=<optimized out>, pp_stack=0x7fffdc044820) at Python/ceval.c:4374 #24 PyEval_EvalFrameEx (f=f@entry=0x36b0c00, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #25 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=<optimized out>, argcount=argcount@entry=4, kws=kws@entry=0x36acac0, kwcount=2, defs=0x7fffedb795a8, defcount=7, closure=0x0) at Python/ceval.c:3584 #26 0x00007ffff7ac8800 in fast_function (nk=<optimized out>, na=4, n=<optimized out>, pp_stack=0x7fffdc044a60, func=<optimized out>) at Python/ceval.c:4447 #27 call_function (oparg=<optimized out>, pp_stack=0x7fffdc044a60) at Python/ceval.c:4372 #28 PyEval_EvalFrameEx (f=f@entry=0x36ac900, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #29 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffdc068928, argcount=3, kws=kws@entry=0x7ffff7fa4068, kwcount=kwcount@entry=0, defs=defs@entry=0x0, ---Type <return> to continue, or q <return> to quit--- defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3584 #30 0x00007ffff7a40835 in function_call (func=0x7fffdc0af140, arg=0x7fffdc068910, kw=0x7fffdc060c58) at Objects/funcobject.c:523 #31 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffdc0af140, arg=arg@entry=0x7fffdc068910, kw=kw@entry=0x7fffdc060c58) at Objects/abstract.c:2547 #32 0x00007ffff7ac38d2 in ext_do_call (nk=<optimized out>, na=<optimized out>, flags=<optimized out>, pp_stack=0x7fffdc044d70, func=0x7fffdc0af140) at Python/ceval.c:4666 #33 PyEval_EvalFrameEx (f=f@entry=0x7fffdc078238, throwflag=throwflag@entry=0) at Python/ceval.c:3028 #34 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffdeac0ea0, argcount=6, kws=kws@entry=0x7ffff7fa4068, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3584 #35 0x00007ffff7a40835 in function_call (func=0x7fffecf8f050, arg=0x7fffdeac0e88, kw=0x7fffdc07ad70) at Objects/funcobject.c:523 #36 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffecf8f050, arg=arg@entry=0x7fffdeac0e88, kw=kw@entry=0x7fffdc07ad70) at Objects/abstract.c:2547 #37 0x00007ffff7ac38d2 in ext_do_call (nk=<optimized out>, na=<optimized out>, flags=<optimized out>, pp_stack=0x7fffdc045070, func=0x7fffecf8f050) at Python/ceval.c:4666 #38 PyEval_EvalFrameEx (f=f@entry=0x7fffdc07b620, throwflag=throwflag@entry=0) at Python/ceval.c:3028 #39 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffdeac0f08, argcount=6, kws=kws@entry=0x7ffff7fa4068, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3584 #40 0x00007ffff7a40835 in function_call (func=0x7fffecf8f230, arg=0x7fffdeac0ef0, kw=0x7fffdc054d70) at Objects/funcobject.c:523 #41 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffecf8f230, arg=arg@entry=0x7fffdeac0ef0, kw=kw@entry=0x7fffdc054d70) at Objects/abstract.c:2547 #42 0x00007ffff7ac38d2 in ext_do_call (nk=<optimized out>, na=<optimized out>, flags=<optimized out>, pp_stack=0x7fffdc045370, func=0x7fffecf8f230) at Python/ceval.c:4666 #43 PyEval_EvalFrameEx (f=f@entry=0x36ab6a0, throwflag=throwflag@entry=0) at Python/ceval.c:3028 #44 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffeb4344e8, argcount=1, kws=kws@entry=0x7ffff7fa4068, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3584 #45 0x00007ffff7a40835 in function_call (func=0x7fffdc07f2a8, arg=0x7fffeb4344d0, kw=0x7fffdc061910) at Objects/funcobject.c:523 #46 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffdc07f2a8, arg=arg@entry=0x7fffeb4344d0, kw=kw@entry=0x7fffdc061910) at Objects/abstract.c:2547 #47 0x00007ffff7ac38d2 in ext_do_call (nk=<optimized out>, na=<optimized out>, flags=<optimized out>, pp_stack=0x7fffdc045670, func=0x7fffdc07f2a8) at Python/ceval.c:4666 #48 PyEval_EvalFrameEx (f=f@entry=0x7fffdc075cc8, throwflag=throwflag@entry=0) at Python/ceval.c:3028 #49 0x00007ffff7ac891b in fast_function (nk=<optimized out>, na=1, n=<optimized out>, pp_stack=0x7fffdc045800, func=<optimized out>) at Python/ceval.c:4437 #50 call_function (oparg=<optimized out>, pp_stack=0x7fffdc045800) at Python/ceval.c:4372 #51 PyEval_EvalFrameEx (f=f@entry=0x36aaa60, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #52 0x00007ffff7ac891b in fast_function (nk=<optimized out>, na=1, n=<optimized out>, pp_stack=0x7fffdc0459a0, func=<optimized out>) at Python/ceval.c:4437 #53 call_function (oparg=<optimized out>, pp_stack=0x7fffdc0459a0) at Python/ceval.c:4372 #54 PyEval_EvalFrameEx (f=f@entry=0x7fffdc076e50, throwflag=throwflag@entry=0) at Python/ceval.c:2989 #55 0x00007ffff7ac9d0d in PyEval_EvalCodeEx (co=<optimized out>, globals=<optimized out>, locals=locals@entry=0x0, args=args@entry=0x7fffedb4e8e8, argcount=1, kws=kws@entry=0x0, kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0, closure=0x0) at Python/ceval.c:3584 #56 0x00007ffff7a40760 in function_call (func=0x7fffed216c08, arg=0x7fffedb4e8d0, kw=0x0) at Objects/funcobject.c:523 #57 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffed216c08, arg=arg@entry=0x7fffedb4e8d0, kw=kw@entry=0x0) at Objects/abstract.c:2547 ---Type <return> to continue, or q <return> to quit--- #58 0x00007ffff7a1cec5 in instancemethod_call (func=0x7fffed216c08, arg=0x7fffedb4e8d0, kw=0x0) at Objects/classobject.c:2602 #59 0x00007ffff7a0e203 in PyObject_Call (func=func@entry=0x7fffdc068370, arg=arg@entry=0x7ffff7fa4050, kw=<optimized out>) at Objects/abstract.c:2547 #60 0x00007ffff7ac01c7 in PyEval_CallObjectWithKeywords (func=0x7fffdc068370, arg=0x7ffff7fa4050, kw=<optimized out>) at Python/ceval.c:4221 #61 0x00007ffff7b0d9c2 in t_bootstrap (boot_raw=0xdef990) at Modules/threadmodule.c:620 #62 0x00007ffff77a4aa1 in start_thread () from /lib64/libpthread.so.0 #63 0x00007ffff6e6693d in clone () from /lib64/libc.so.6 Core dump available upon request ---------- components: Interpreter Core messages: 298650 nosy: Thomas Mortensson priority: normal severity: normal status: open title: Python core crash during os.fork() on CentOS 6.5 (Python 2.7.13) type: crash versions: Python 2.7 _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30967> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com