[zeromq-dev] multiprocessing forked children kill zeromq server
We have a system which uses Python multiprocessing to run multiple simulations concurrently in child processes, with main server - child process communication using multiprocessing. This works fine standalone. For a GUI, we use zeromq and tornado for communications between the main server and the browser. This also works fine as long as the main server never starts a child process using multiprocessing. If we do start a child process, we get an assertion failure: Assertion failed: ok (bundled/zeromq/src/mailbox.cpp:79) and the main server aborts (this is on Linux, Windows seems OK, probably because multiprocessing doesn't fork on Windows) This is using Python 2.7 and pyzmq 13.1.0. Is there some way to run a server that uses zmq and also forks child processes using multiprocessing? I've seen some posts about forked child processes needing to context.term() and then creating a new context. I tried context.term() in the target of the multiprocess.Process, but that hasn't fixed the problem. Note that the child processes do not need zmq, so I don't create a new context. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] multiprocessing forked children kill zeromq server
MInRk noted that: You cannot continue to use zmq sockets after a fork - you have to take care in your application that no sockets created before the fork will be [not be] used by any calls in the child process. Thanks, that makes sense. But can anyone provide guidance in how to do that? I'm quite new to zmq, but somewhat versed in Python multiprocessing. I think there's only one zmq context in use by the main server, and in the child process I context.term() as soon as I can. That did not fix the issue (but then I don't know that it should). It's entirely possible that there are zmq messages in flight while the multiprocessing fork is going on, so I can imagine that when a child 'wakes up' it might think it should process a message that it shouldn't. Is it possible to 'suspend' zmq while the fork is occurring so I can somehow have the child cleanup things it shouldn't be using, and then the parent resumes zmq processing? Or is there some way to mark the sockets such that they aren't inherited by the child? Or something else I can try? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
