On Monday, March 24, 2014 7:19:56 PM UTC-5, Chris Angelico wrote: > On Tue, Mar 25, 2014 at 7:24 AM, Matt Newville > > > I'm maintaining a python interface to a C library for a distributed > > control system (EPICS, sort of a SCADA system) that does a large > > amount of relatively light-weight network I/O. In order to keep many > > connections open and responsive, and to provide a simple interface, > > the python library keeps a global store of connection state. > > > > This works well for single processes and threads, but not so well for > > multiprocessing, where the global state causes trouble. > > > From the sound of things, a single process is probably what you want > here. Is there something you can't handle with one process?
Thanks for the reply. I find that appreciation is greatly (perhaps infinitely) delayed whenever I reply "X is probably not what you want to do" without further explanation to a question of "can I get some advice on how to do X?". So, I do thank you for your willingness to reply, even such a guaranteed-to-be-under-appreciated reply. There are indeed operations that can't be handled with a single process, such as simultaneously using multiple cores. This is why we want to use multiprocessing instead of (or, in addition to) threading. We're trying to do real-time collection of scientific data from a variety of data sources, generally within a LAN. The data can get largish and fast, and intermediate processing occasionally requires non-trivial computation time. So being able to launch worker processes that can run independently on separate cores would be very helpful. Ideally, we'd like to let sub-processes make calls to the control system too, say, read new data. I wasn't really asking "is multiprocessing appropriate?" but whether there was a cleaner way to subclass multiprocessing.BaseManager() to use a subclass of Process(). I can believe the answer is No, but thought I'd ask. Thanks again, --Matt -- https://mail.python.org/mailman/listinfo/python-list
