> > > It looks like that SO answer is moving data into the global scope of each > worker. It is probably worth experimenting with but I'd be worried about > performance implications of non-const global variables. It's probably the > case that this is still a win for my use case though. Thanks for the link. >
Why not declare input vector as const? I have a similar requirement for a simulation. I found it convenient to wrap everything that is required on workers into a module. module SphericalHarmonicTransforms export spherical_harmonic_transforms const input = Vector{Float64}(10^7) ... function spherical_harmonic_transforms(idx) coefficients = Vector{Complex128}(10^6) ... # reference input vector here return coefficients end end Then to propagate to all workers, just write using SphericalHarmonicTransforms function sim() idx = 1 limit = 10000 nextidx() = (myidx = idx; idx += 1; myidx) @sync for worker in workers() @async while true myidx = nextidx() myidx ≤ limit || break coefficients = remotecall_fetch(worker, spherical_harmonic_transforms, myidx) write_results_to_disk(coefficients) end end end addprocs() using SphericalHarmonicTransforms sim()