Joshua Landau wrote: > On 7 August 2013 11:10, Luca Cerone <luca.cer...@gmail.com> wrote: >> I can't try it now, I'll let you know later if it works! >> (Though just by reading I can't really understand what the code does). > > Well, > >>> from multiprocessing import Pool >>> from functools import partial >>> >>> class A(object): >>> def __init__(self,x): >>> self.value = x >>> def fun(self,x): >>> return self.value**x > > This is all the same, as with > >>> l = range(100) >>> p = Pool(4) > > You then wanted to do: > >> op = p.map(A(3).fun, l) > > but bound methods can't be pickled, it seems. > > However, A(3) *can* be pickled. So what we want is a function: > > def proxy(arg): > A(3).fun(arg) > > so we can write: > >> op = p.map(proxy, l) > > To generalise you might be tempted to write: > > def generic_proxy(instance, name): > def proxy(arg): > # Equiv. of instance.name(arg) > getattr(instance, name)(arg) > > but the inner function won't work as functions-in-functions can't be > pickled either. > > So we use: > >>> def _getattr_proxy_partialable(instance, name, arg): >>> return getattr(instance, name)(arg) > > Which takes all instance, name and arg. Of course we only want our > function to take arg, so we partial it: > >>> def getattr_proxy(instance, name): >>> """ >>> A version of getattr that returns a proxy function that can >>> be pickled. Only function calls will work on the proxy. >>> """ >>> return partial(_getattr_proxy_partialable, instance, name) > > partial objects are picklable, btw. > >>> op = p.map(getattr_proxy(A(3), "fun"), l) >>> print(op) > > :)
There is also the copy_reg module. Adapting <http://mail.python.org/pipermail/python-list/2008-July/469164.html> you get: import copy_reg import multiprocessing import new def make_instancemethod(inst, methodname): return getattr(inst, methodname) def pickle_instancemethod(method): return make_instancemethod, (method.im_self, method.im_func.__name__) copy_reg.pickle( new.instancemethod, pickle_instancemethod, make_instancemethod) class A(object): def __init__(self, a): self.a = a def fun(self, b): return self.a**b if __name__ == "__main__": items = range(10) pool = multiprocessing.Pool(4) print pool.map(A(3).fun, items) -- http://mail.python.org/mailman/listinfo/python-list