No. I've thought about maybe writing a wrapper for Spark but only after
julia 0.4 (and the new and improved dataframes) have landed. Also depends
how much time I could spend on it at my day job :)
On Friday, May 1, 2015 at 2:57:57 PM UTC+2, Sebastian Good wrote:
>
> Steven, are you working on su
Steven, are you working on such things at the moment?
On Friday, May 1, 2015 at 4:50:39 AM UTC-4, Steven Sagaert wrote:
>
> I'd be nice to see a distributed array implemented on top of MPI (or
> similar high perf distribution libs) like Fortran co-arrays but since I'm
> out of academia and do
I'd be nice to see a distributed array implemented on top of MPI (or
similar high perf distribution libs) like Fortran co-arrays but since I'm
out of academia and do not have access to "real" supercomputers anymore
I'm actually more interested in wrappers to cloud base distributed
computing fr
Yes, performance will be largely the same on 0.4.
If you have to do any performance sensitive code at scale MPI is really the
only option I can recomend now. I don't know what you are trying to do but
the MPI.jl library is a bit incomplete so it would be great if you used it
and could contribu
Hi Jake,
Jake Bolewski writes:
> DistributedArray performance is pretty bad. The reason for removing
> them from base was to spur their development. All I can say at this
> time is that we are actively working on making their performance
> better.
OK, thanks. Should I try with the DistributedA
Also, you want to map(fetch, refs) not pmap.
With that i get better speedup (still not great, but at least > 2x with 8
processors)
julia> N=100;T=1000;A=rand(3,N);@time SimulationSerial(A,N,T)
elapsed time: 1.822478028 seconds (233 kB allocated)
julia> N=100;T=1000;dA=drand(3,N);@time
DistributedArray performance is pretty bad. The reason for removing them
from base was to spur their development. All I can say at this time is
that we are actively working on making their performance better.
For every parallel program you have implicit serial overhead (this is
especially tru
Hi,
I can't say anything regarding the performance of Distributed Arrays.
However, note that they have been relocated from Base to a separate
package: https://github.com/JuliaParallel/DistributedArrays.jl which should
work with 0.4-dev.
Best,
Alex.
On Thursday, 30 April 2015 12:10:47 UTC+2,
