On 2016-10-26 21:44, pic8...@gmail.com wrote:
On Monday, October 24, 2016 at 12:39:47 PM UTC-5, Thomas Nyberg wrote:
On 10/24/2016 12:45 PM, pic8...@gmail.com wrote:
> Thanks for the reply.
>
> The code snippet given by Peter is not very clear
>
> I would like to multiprocess a function which is written in python of the
form bar(**kwargs) which returns a value. This example does not return anything
>
> Would you please use my example for the map function?
>
> I appreciate your help,
>
I'm honestly not totally sure what you want to do. However, say you want
to do the following (btw this is basically what Dennis said i nhis last
email, but maybe I can help clarify):
kwargs = {'param1': val1, 'param2': val2})
Then you'd like to have the following two operations performed in
separate processes:
bar(param1=val1)
bar(param2=val2)
In that case, I guess I would do something like the following. First
define bar_wrapper as follows (*I haven't tested any code here!):
def bar_wrapper(pair):
key, val = pair
return bar(**{key: val})
Then I would probably do something like
map(bar_wrapper, kwargs.items())
I.e. basically what I'm doing is taking the key-val pairs and producing
a list of them (as tuples). This is something that you can apply map
too, but not with the original function. So then the wrapper function
converts the tuple back to what you want originally.
Hopefully I'm understanding correctly and hopefully this helps.
Cheers,
Thomas
Thomas,
I have strings & numpy.ndarray as arguments. The wrapper function works great
for strings.
Here's what I'm trying to do...
**************************
I have a serial python fn of the form
def bar(**kwargs):
a=kwargs.get("name")
print a
self.a1d=ma.asanyarray(kwargs.get('a1d'), dtype=float)
(****more calculations--takes a long time to compute
returns an object)
I am trying to run this function in parallel.
Here's my Main program
import numpy as np
import numpy.ma as ma
from delegate import parallelize
from hashlib import sha1
a1d=np.zeros((32))
b1d=np.zeros((32))
p=open("/tmp/pdata","rb")
pdata=np.load(p)
for i in range(0,10):
a1d=pdata['t1d']
b1d=pdata['gz1d']
print a1d,b1d
kwargs={'name':'special','a':a1d,'b':b1d}
val=parallelize(bar,kwargs)
***************************************
Error:
line 30, in <module>
val=parallelize(bar,kwargs)
delegate.py", line 311, in process
item, items = items[0], items[1:]
KeyError: 0
**************************************
Error: (with the wrapper function)
val=parallelize(bar_wrapper,kwargs.items())
TypeError: unhashable type: 'numpy.ndarray'
***************************************
How do I pass the string & numpy.ndarray to the function bar(**kwargs)?
Thank you for suggestions & help,
a1d=np.zeros((32))
b1d=np.zeros((32))
p=open("/tmp/pdata","rb")
pdata=np.load(p)
for i in range(0,100):
a1d=pdata['t1d']
b1d=pdata['gz1d']
print a1d,b1d
kwargs={'name':'special','a':a1d,'b':b1d}
val=parallelize(bar,kwargs)
'parallelize' expects a function and a list of items. It calls the
function with each item in the list in parallel, passing the item as the
argument of the function.
That _single_ argument item can be a tuple or a dict.
--
https://mail.python.org/mailman/listinfo/python-list
