New submission from Albert Zeyer <alb...@googlemail.com>: subprocess_fork_exec currently calls fork().
I propose to use vfork() or posix_spawn() or syscall(SYS_clone, SIGCHLD, 0) instead if possible and if there is no preexec_fn. The difference would be that fork() will call any atfork handlers (registered via pthread_atfork()), while the suggested calls would not. There are cases where atfork handlers are registered which are not save to be called e.g. in multi-threaded environments. In the case of subprocess_fork_exec without preexec_fn, there is no need to call those atfork handlers, so avoiding this could avoid potential problems. It's maybe acceptable if a pure fork() without exec() doesn't work in this case anymore, but there is no reason that a fork()+exec() should not work in any such cases. This is fixed by my proposed solution. An example case is OpenBlas and OpenMP, which registers an atfork handler which is safe to be called if there are other threads running. See here: https://github.com/tensorflow/tensorflow/issues/13802 https://github.com/xianyi/OpenBLAS/issues/240 https://trac.sagemath.org/ticket/22021 About fork+exec without the atfork handlers, see here for alternatives (like vfork): https://stackoverflow.com/questions/46810597/forkexec-without-atfork-handlers/ ---------- components: Interpreter Core messages: 304587 nosy: Albert.Zeyer priority: normal severity: normal status: open title: subprocess_fork_exec more stable with vfork type: behavior versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31814> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
