He Andrew,
I used @parallel in sage to parallelize some tensorial calculus (for
example : src/sage/tensor/modules/comp.py)
It is not very complicate to use it but sometimes you need to reorganize
your computation.
The parallelism depends strictly on your code so the part of code you
posted is not enough to understand how parallelize it.
The main action you have to to is:
1) Create a function (with @parallel decoration) which will be used by your
cores to computes a part of the computation :
ex:
@parallel(p_iter='multiprocessing',ncpus=24)
def paral_prod(myfunc,local_list_ind):
partial = []
# local_list_ind is a subset of set of "s" and "t" pairs
for ind in local_list_ind :
# ind[0] is the "s", ind[1] is the "t" of your code
partial.append([ind[0],ind[0],myfunc(ind[0],ind[0]))
return partial
Note that multiprocessing does not use really shared memory so you have to
put in your function argument all the variable you need.
2) Then create a "list" of input (so of "tuple"):
ex :
listParalInput =[(self._inner_product_st,[[1,2],[1,3]]),(self.
_inner_product_st,[[2,3],[1,3]],.......) ]
In this example any tuple in the list will be send to a core.
Then the first core will execute :
paral_prod(self._inner_product_st,[[1,2],[1,3]])
and so on for the other cores.
3) The you can call the parallel function and you have to put the result in
"mat"
for ii,val in paral_prod(listParalInput):
for jj in val:
# any jj in val contains : [ s ,t ,
self._inner_product_st(1,2)]
mat[jj[0],jj[1]] = jj[2]
mat[jj[1],jj[0]] = jj[2]
I have not tested this code but it should work (at least in principle).
It remains to create the listParalInput automatically (so you have to do a
partition of the indices of the triangular matrix).
I have also a little doubt concerning the use of self._inner_product_st as
an argument, but you should test it.
Regards,
Marco
Le mercredi 11 mai 2016 08:13:47 UTC+2, Andrew a écrit :
>
> Thanks Vincent for both your answers. I am planning on using just one
> machine with up to 24 cores. (If I am not able to get far enough this way
> then I will think about using the cluster. Using 24 cores will already be a
> big step up compared with what I am currently doing.)
>
> If the processes can share memory that would probably be good, but it is
> not essential. As this seems to require extra effort I'll try first to see
> how it goes without sharing.
>
> Andrew
>
>
> On Wednesday, 11 May 2016 14:56:27 UTC+10, vdelecroix wrote:
>>
>> On 10/05/16 23:32, Andrew wrote:
>> > On Wednesday, 11 May 2016 14:00:20 UTC+10, William wrote:
>> >>
>> >>
>> >> It depends on what you are trying to do. For some computations,
>> >> @parallel is the best possible way to do them, and for others it is
>> >> the worst possible way.
>> >>
>> >> Is there a good to learn about how best to do this (in
>> sage/python/...)?
>> >
>> > The first example that I care about looks something like this:
>> >
>> > mat=[[0 for s in xrange(tabs)] for t in xrange(tabs)]
>> > for s in xrange(tabs):
>> > for t in xrange(s,tabs):
>> > mat[s][t]=self._inner_product_st(s,t)
>> > mat[t][s]=mat[s][t]
>> >
>> > The `_inner_product_st` method computes certain structure constants in
>> a
>> > module. This method is time consuming and slightly recursive with
>> "basic"
>> > cases being cached. Parallelising this loop seemed like the right place
>> to
>> > me but, to be honest, I have no idea what I am doing!
>>
>> At this point it is not clear to me whether you need shared memory or
>> not between the processes. This can be an important overhead of
>> parallelization.
>>
>
--
You received this message because you are subscribed to the Google Groups
"sage-support" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sage-support+unsubscr...@googlegroups.com.
To post to this group, send email to sage-support@googlegroups.com.
Visit this group at https://groups.google.com/group/sage-support.
For more options, visit https://groups.google.com/d/optout.
