Done, see https://github.com/JuliaLang/julia/issues/17509
On Wednesday, July 20, 2016 at 5:21:23 PM UTC+2, Cedric St-Jean wrote: > > That does look suspicious. Maybe file an issue if there isn't one? > > On Wed, Jul 20, 2016 at 4:31 AM, Marius Millea <marius...@gmail.com > <javascript:>> wrote: > >> I don't think that theory totally works, it seems to scale to some extent >> with the length of time to load the package itself. Another example: >> >> julia> tic(); using PyPlot; toc() >> elapsed time: 3.395904233 seconds >> >> vs >> >> julia> addprocs(); >> julia> tic(); using PyPlot; toc() >> elapsed time: 13.877550518 seconds >> >> or even: >> >> julia> addprocs(); >> julia> using Empty; tic(); using PyPlot; toc() >> elapsed time: 7.357315778 seconds >> >> >> In any case, it can get pretty painful loading a few modules at the >> beginning of my parallelized scripts... >> >> >> >> >> On Tuesday, July 19, 2016 at 4:55:40 PM UTC+2, Cedric St-Jean wrote: >>> >>> Yes, that's what I meant. Presumably the multi-proc machinery is getting >>> compiled at the first `using`. It's the same reason why "println(2+2)" is >>> very slow on first use, but fast afterwards. >>> >>> On Tue, Jul 19, 2016 at 10:41 AM, Marius Millea <marius...@gmail.com> >>> wrote: >>> >>>> Seems it may have something to do with that. If I understood correctly >>>> what you're saying, if I create Empty2.jl defining module Empty2, I get, >>>> >>>> julia> addprocs(); >>>> >>>> julia> tic(); using Empty; toc() >>>> elapsed time: 2.706353202 seconds >>>> 2.706353202 >>>> >>>> julia> tic(); using Empty; toc() >>>> elapsed time: 0.00042397 seconds >>>> 0.00042397 >>>> >>>> julia> tic(); using Empty2; toc() >>>> elapsed time: 0.029200919 seconds >>>> 0.029200919 >>>> >>>> julia> tic(); using Empty2; toc() >>>> elapsed time: 0.000193097 seconds >>>> 0.000193097 >>>> >>>> >>>> >>>> That first load of Empty2 at 0.02 secs is much more in line with what >>>> loading it on a single processor takes. >>>> >>>> >>>> >>>> On Tuesday, July 19, 2016 at 4:13:15 PM UTC+2, Cedric St-Jean wrote: >>>>> >>>>> Maybe there is some warm-up JIT time in there? If you create an Empty2 >>>>> module and load it after Empty, is it also slow? >>>>> >>>>> On Tuesday, July 19, 2016 at 9:07:01 AM UTC-4, Marius Millea wrote: >>>>>> >>>>>> I noticed that once I addprocs(), subsequent "using" statements were >>>>>> extremely slow. I guess in this case its loading the module on each >>>>>> processor, but if it happens in parallel it shouldn't be *that* much >>>>>> more >>>>>> wall time, and here I'm talking about two orders of magnitude >>>>>> difference. >>>>>> >>>>>> Assuming I've got a file Empty.jl who contents is, >>>>>> >>>>>> module Empty >>>>>> end >>>>>> >>>>>> then single threaded: >>>>>> >>>>>> tic() >>>>>> using Empty >>>>>> toc() >>>>>> elapsed time: 0.024461076 seconds >>>>>> >>>>>> vs. multi-threaded: >>>>>> >>>>>> addprocs() #I've got 8 procs >>>>>> tic() >>>>>> using Empty >>>>>> toc() >>>>>> elapsed time: 2.479418079 seconds >>>>>> >>>>>> >>>>>> Should I submit this as an Issue on Github, or is there something >>>>>> else going on? I've checked both Julia 0.4.5. and 0.5 (01e3c8a). I'm on >>>>>> Ubuntu 16.04 64bit. >>>>>> >>>>>> >>>>>> >>> >
