mheavner schrieb:
I'm using multiprocessing to spawn several subprocesses, each of which
uses a very large data structure (making it impractical to pass it via
pipes / pickling). I need to allocate this structure once when the
process is created and have it remain in memory for the duration of
the process. The way the multiprocessing module is set up, only the
'run' method runs within the subprocess - so creating a wrapper class
with a constructor that allocates the structure in __init__ will not
work, as far as I know, as this will still be within the parent
process.

If I were working in C/C++, I would declare the variable "static"
within the function body - is there any way with the multiprocessing
module to have persistent data members within subprocesses?

Works for me, at least under OSX (and I presume *nixes in general work.) No idea about Windows.


The thing to keep in mind is that forking is used, and that interpreter-state up to the moment of the fork is the same for all subprocesses.



from multiprocessing import Process

class MyProcess(Process):


    def __init__(self, huge_shared_state):
        self.huge_shared_state = huge_shared_state
        super(MyProcess, self).__init__()


    def run(self):
        print self.name, len(self.huge_shared_state)




shared_state = range(1000000)

processes = []
for i in xrange(10):
    p = MyProcess(shared_state)
    p.start()
    processes.append(p)


for p in processes:
    p.join()



Diez
--
http://mail.python.org/mailman/listinfo/python-list

Reply via email to