On Tue, Nov 30, 2010 at 1:35 PM, Valery Khamenya <khame...@gmail.com> wrote: > Hi, > > multithreading.pool Pool has a promissing initializer argument in its > constructor. > However it doesn't look possible to use it to initialize each Pool's > worker with some individual value (I'd wish to be wrong here) > > So, how to initialize each multithreading Pool worker with the > individual values? > > The typical use case might be a connection pool, say, of 3 workers, > where each of 3 workers has its own TCP/IP port. > > from multiprocessing.pool import Pool > > def port_initializer(_port): > global port > port = _port > > def use_connection(some_packet): > global _port > print "sending data over port # %s" % port > > if __name__ == "__main__": > ports=((4001,4002, 4003), ) > p = Pool(3, port_initializer, ports) # oops... :-) > some_data_to_send = range(20) > p.map(use_connection, some_data_to_send)
Using an initializer with multiprocessing is something I've never tried. I have used queues with multiprocessing though, and I believe you could use them, at least as a fallback plan if you prefer to get the initializer to work. If you create in the parent a queue in shared memory (multiprocessing facilitates this nicely), and fill that queue with the values in your ports tuple, then you could have each child in the worker pool extract a single value from this queue so each worker can have its own, unique port value. HTH -- http://mail.python.org/mailman/listinfo/python-list