That does look suspicious. Maybe file an issue if there isn't one? On Wed, Jul 20, 2016 at 4:31 AM, Marius Millea <mariusmil...@gmail.com> wrote:
> I don't think that theory totally works, it seems to scale to some extent > with the length of time to load the package itself. Another example: > > julia> tic(); using PyPlot; toc() > elapsed time: 3.395904233 seconds > > vs > > julia> addprocs(); > julia> tic(); using PyPlot; toc() > elapsed time: 13.877550518 seconds > > or even: > > julia> addprocs(); > julia> using Empty; tic(); using PyPlot; toc() > elapsed time: 7.357315778 seconds > > > In any case, it can get pretty painful loading a few modules at the > beginning of my parallelized scripts... > > > > > On Tuesday, July 19, 2016 at 4:55:40 PM UTC+2, Cedric St-Jean wrote: >> >> Yes, that's what I meant. Presumably the multi-proc machinery is getting >> compiled at the first `using`. It's the same reason why "println(2+2)" is >> very slow on first use, but fast afterwards. >> >> On Tue, Jul 19, 2016 at 10:41 AM, Marius Millea <marius...@gmail.com> >> wrote: >> >>> Seems it may have something to do with that. If I understood correctly >>> what you're saying, if I create Empty2.jl defining module Empty2, I get, >>> >>> julia> addprocs(); >>> >>> julia> tic(); using Empty; toc() >>> elapsed time: 2.706353202 seconds >>> 2.706353202 >>> >>> julia> tic(); using Empty; toc() >>> elapsed time: 0.00042397 seconds >>> 0.00042397 >>> >>> julia> tic(); using Empty2; toc() >>> elapsed time: 0.029200919 seconds >>> 0.029200919 >>> >>> julia> tic(); using Empty2; toc() >>> elapsed time: 0.000193097 seconds >>> 0.000193097 >>> >>> >>> >>> That first load of Empty2 at 0.02 secs is much more in line with what >>> loading it on a single processor takes. >>> >>> >>> >>> On Tuesday, July 19, 2016 at 4:13:15 PM UTC+2, Cedric St-Jean wrote: >>>> >>>> Maybe there is some warm-up JIT time in there? If you create an Empty2 >>>> module and load it after Empty, is it also slow? >>>> >>>> On Tuesday, July 19, 2016 at 9:07:01 AM UTC-4, Marius Millea wrote: >>>>> >>>>> I noticed that once I addprocs(), subsequent "using" statements were >>>>> extremely slow. I guess in this case its loading the module on each >>>>> processor, but if it happens in parallel it shouldn't be *that* much more >>>>> wall time, and here I'm talking about two orders of magnitude difference. >>>>> >>>>> Assuming I've got a file Empty.jl who contents is, >>>>> >>>>> module Empty >>>>> end >>>>> >>>>> then single threaded: >>>>> >>>>> tic() >>>>> using Empty >>>>> toc() >>>>> elapsed time: 0.024461076 seconds >>>>> >>>>> vs. multi-threaded: >>>>> >>>>> addprocs() #I've got 8 procs >>>>> tic() >>>>> using Empty >>>>> toc() >>>>> elapsed time: 2.479418079 seconds >>>>> >>>>> >>>>> Should I submit this as an Issue on Github, or is there something else >>>>> going on? I've checked both Julia 0.4.5. and 0.5 (01e3c8a). I'm on Ubuntu >>>>> 16.04 64bit. >>>>> >>>>> >>>>> >>
