Re: [julia-users] Best way to make a module available to all pmap workers?

2016-04-05 Thread Tim Holy
Do you need the @everywhere? using should be good enough on its own.

However, this isn't working perfectly: see
https://github.com/JuliaLang/julia/issues/9245
https://github.com/JuliaLang/julia/issues/3674

for some things you may need to work around (or better yet, submit a fix for!). 
In particular, your experience and #9245 can both be worked around with the 
following "amusing" sequence:

$ julia -p 2
   _
   _   _ _(_)_ |  A fresh approach to technical computing
  (_) | (_) (_)|  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "?help" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.4-pre+2 (2016-01-18 02:17 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit a85c3a0* (78 days old release-0.4)
|__/   |  x86_64-linux-gnu

julia> using ColorTypes

julia> @everywhere using ColorTypes


The first one loads the code on all workers; the only thing the second one does 
is to bring the bindings into Main (if you need that).

Best,
--Tim

On Tuesday, April 05, 2016 06:36:22 AM Thomas Covert wrote:
> I have some code in a module (say, MyModule.jl) for which I would like each
> worker in a pmap() to have access.  Right now, I accomplish this by
> something like:
> 
> addprocs(CPU_CORES)
> @everywhere using MyModule
> function dostuff(x)
> # code that uses functions exported from MyModule
> end
> results = pmap(dostuff, X)
> 
> where X is an array of things to pass to dostuff
> 
> This works.  However, the @everywhere macro called on the using statement
> results in this set of warnings:
> 
> WARNING: replacing module MyModule
> 
> WARNING: replacing module MyModule
> 
> WARNING: replacing module MyModule
> 
> WARNING: replacing module MyModule
> 
> 
> Is this the idiomatic way to accomplish this task?  I would have thought
> there is a way to write this code so that warnings such as these don't
> appear.



[julia-users] Best way to make a module available to all pmap workers?

2016-04-05 Thread Thomas Covert
I have some code in a module (say, MyModule.jl) for which I would like each 
worker in a pmap() to have access.  Right now, I accomplish this by 
something like:

addprocs(CPU_CORES)
@everywhere using MyModule
function dostuff(x)
# code that uses functions exported from MyModule
end
results = pmap(dostuff, X)

where X is an array of things to pass to dostuff

This works.  However, the @everywhere macro called on the using statement 
results in this set of warnings:

WARNING: replacing module MyModule

WARNING: replacing module MyModule

WARNING: replacing module MyModule

WARNING: replacing module MyModule


Is this the idiomatic way to accomplish this task?  I would have thought 
there is a way to write this code so that warnings such as these don't 
appear.