I also noticed (not specific to Power machines) that when you want to load packages in Julia with several workers, I have to do the following to have successful loading of packages:
nprocs()==1 && addprocs()
using Knet
@everywhere using Knet
using RDatasets
@everywhere using RDatasets
using MLBase
@everywhere using MLBase
If I use,
nprocs()==1 && addprocs()
@everywhere using RDatasets
@everywhere using Knet
@everywhere using MLBase
I have this error:
ERROR: On worker 2:
LoadError: LoadError: UndefVarError: @knet not defined
If I use,
nprocs()==1 && addprocs()
using RDatasets
using Knet
using MLBase
ERROR (unhandled task failure): On worker 4:
On worker 4:
On worker 4:
UndefVarError: Gaussian not defined
Attached is the code that I use which uses 3 @parallel for syntax
My insight is that you need to load first all the packages into the process
id 1 to avoid clashes during pre-cache compilation and use @everywhere
using
to export the functions to other processes. It seems that just *using*
directives does not immediately export those functions to all processes
based on the code attached such that I have to use "@everywhere using"
directives. Is this a bug?
Please try to reproduce the errors I have with the attached code. This
error only happens if you do addprocs() to run parallel julia.
I'm using Julia 0.4.6 (2016-06-19 17:16 UTC)
On Fri, Sep 2, 2016 at 9:52 AM, Paulito Palmes <ppal...@gmail.com> wrote:
> Hi All,
>
> I noticed that some of the packages are not aware that Julia has power
> port now and when you install those packages, they include some compiler
> directives specific to Intel processor such as SSE. In the Makefile, I
> removed this extra option and the package compiles. However, if I do
> Pkg.build("Package"), it does not respect the edited Makefile and starts to
> download from github. I don't know the sequence of operations behind but is
> there a way to manually compile and install the local package downloaded
> after you edit some files to make it work for the power machine?
>
> I think the long term solution is to have a test server running power
> machine to flag that the package did not compile successfully to the power
> port.
>
> -paulito
>
> On Fri, Sep 2, 2016 at 8:02 AM, Viral Shah <vi...@mayin.org> wrote:
>
>> Can you try this one? I believe this error is addressed now.
>>
>> http://s3.amazonaws.com/julianightlies/bin/linux/ppc64le/
>> julia-latest-linuxppc64.tar.gz
>>
>> Also, Elliot enabled the nightlies:
>>
>> https://build.julialang.org/builders/package_tarballppc64le
>>
>> -viral
>>
>>
>> > On Sep 2, 2016, at 4:15 AM, James Fairbanks <jpfairba...@gmail.com>
>> wrote:
>> >
>> > Hi Viral,
>> >
>> > I got negative results on my power8 machine.
>> > After untarring the link above I got the following errors when just
>> running the repl.
>> >
>> >
>> > [jpf@power8 julia-3005940a21]$ ./bin/julia
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > 'powerpc64le' is not a recognized processor for this target (ignoring
>> processor)
>> > _
>> > _ _ _(_)_ | A fresh approach to technical computing
>> > (_) | (_) (_) | Documentation: http://docs.julialang.org
>> > _ _ _| |_ __ _ | Type "?help" for help.
>> > | | | | | | |/ _` | |
>> > | | |_| | | | (_| | | Version 0.5.0-rc3+3 (2016-08-26 06:19 UTC)
>> > _/ |\__'_|_|_|\__'_| | sf/ppc64le/3005940 (fork: 3 commits, 9 days)
>> > |__/ | powerpc64le-unknown-linux-gnu
>> >
>> >
>> > Then I got a lot more errors.
>> > For example:
>> >
>> > WARNING: Method definition f(Tuple{Vararg{Int64, #N<:Any}},
>> AbstractArray{#T<:Any, #N<:Any}) in module Main at
>> /home/jpf/julia-3005940a21/share/julia/test/core.jl:706 overwritten at
>> /home/jpf/julia-3005940a21/share/julia/test/core.jl:712.
>> > From worker 13: * linalg/diagonal in 101.62 seconds,
>> maxrss 348.31 MB
>> > From worker 13: * inference in 0.92 seconds,
>> maxrss 352.38 MB
>> > From worker 13: * keywordargs in 1.56 seconds,
>> maxrss 354.13 MB
>> > WARNING: Method definition f() in module JLCall14301 at
>> /home/jpf/julia-3005940a21/share/julia/test/core.jl:3529 overwritten at
>> /home/jpf/julia-3005940a21/share/julia/test/core.jl:3539.
>> > From worker 5: * linalg/matmul in 169.17 seconds,
>> maxrss 367.69 MB
>> > From worker 16: * linalg/cholesky in 96.71 seconds,
>> maxrss 332.44 MB
>> > From worker 16: * char Error During Test
>> > From worker 16: Test threw an exception of type InexactError
>> > From worker 16: Expression: $(Expr(:escape,
>> :(convert(Char,Float16(x))))) $(Expr(:escape, :(==))) $(Expr(:escape,
>> :(convert(Char,Float32(x))))) $(Expr(:escape, :(==))) $(Expr(:escape,
>> :(convert(Char,Float64(x))))) $(Expr(:escape, :(==))) $(Expr(:escape,
>> :(Char(x))))
>> > From worker 16: InexactError()
>> > From worker 16: in macro expansion; at
>> /home/jpf/julia-3005940a21/share/julia/test/char.jl:72 [inlined]
>> > From worker 16: in anonymous at ./<missing>:?
>> > From worker 16: in include_string(::String, ::String) at
>> ./loading.jl:380
>> > From worker 16: in include_from_node1(::String) at
>> ./loading.jl:429
>> > From worker 16: in macro expansion at ./util.jl:226 [inlined]
>> > From worker 16: in runtests(::String) at
>> /home/jpf/julia-3005940a21/share/julia/test/testdefs.jl:7
>> > From worker 16: in
>> > (::Base.Serializer.__deserialized_types__.##16#24)(::String)
>> at /home/jpf/julia-3005940a21/share/julia/test/runtests.jl:44
>> > From worker 16: in
>> > (::Base.##625#627{Base.CallMsg{:call_fetch}})()
>> at ./multi.jl:1421
>> > From worker 16: in
>> > run_work_thunk(::Base.##625#627{Base.CallMsg{:call_fetch}},
>> ::Bool) at ./multi.jl:1001
>> > From worker 16: in macro expansion at ./multi.jl:1421
>> [inlined]
>> > From worker 16: in (::Base.##624#626{Base.CallMsg
>> {:call_fetch},Base.MsgHeader,TCPSocket})() at ./event.jl:68
>> >
>> > On Tuesday, August 30, 2016 at 1:30:54 AM UTC-4, Viral Shah wrote:
>> > I should point out that the linalg tests are expected to fail for now,
>> since we are awaiting a new openblas release, which is known to fix these
>> issues.
>> >
>> > -viral
>> >
>> > On Friday, August 19, 2016 at 10:26:38 AM UTC+5:30, Viral Shah wrote:
>> > I have uploaded Julia-0.5 on Power8 binaries here. These are built with
>> the latest openblas (that passes all julia tests) and hence there is no
>> need to use ATLAS.
>> >
>> > https://drive.google.com/open?id=0B0rXlkvSbIfhVWpZb2hqclBIVms
>> >
>> > Would be great if people can try this out.
>> >
>> > -viral
>> >
>> >
>> >
>> > > On Aug 19, 2016, at 9:06 AM, Viral Shah <vi...@mayin.org> wrote:
>> > >
>> > > I am getting successful builds on the OSU Power8 machine. Once
>> openblas has a new release, I suspect we can provide pre-packaged power8
>> binaries.
>> > >
>> > > I am building on CentOS 7 and this is what lscpu says:
>> > >
>> > > Architecture: ppc64le
>> > > Byte Order: Little Endian
>> > > CPU(s): 160
>> > > On-line CPU(s) list: 0-159
>> > > Thread(s) per core: 8
>> > > Core(s) per socket: 10
>> > > Socket(s): 2
>> > > NUMA node(s): 2
>> > > Model: 8335-GCA
>> > > L1d cache: 64K
>> > > L1i cache: 32K
>> > > L2 cache: 512K
>> > > L3 cache: 8192K
>> > > NUMA node0 CPU(s): 0-79
>> > > NUMA node8 CPU(s): 80-159
>> > >
>> > > I suspect you are running into a codegen issue. Let’s give it a few
>> days and once llvm 3.9 releases, we can try that. If not, we may need
>> Jameson’s help.
>> > >
>> > > -viral
>> > >
>> > >
>> > >
>> > >> On Aug 19, 2016, at 8:54 AM, Viral Shah <vi...@mayin.org> wrote:
>> > >>
>> > >> It is hard to say what is happening there. Trying it out.
>> > >>
>> > >> OpenBLAS develop branch is now passing on Power and hence Atlas
>> should no longer be required. Note that you need the latest (3.10.4?)
>> ATLAS, but in any case, this should not affect the build.
>> > >>
>> > >> -viral
>> > >>
>> > >>
>> > >>
>> > >>> On Aug 18, 2016, at 7:03 PM, Geert Janssen <gee...@gmail.com>
>> wrote:
>> > >>>
>> > >>> Dear Viral,
>> > >>>
>> > >>> I am building Julie release-0.5 on a Power8E Ubuntu 14.10 machine.
>> > >>> I use the following Make.user after having had trouble compiling
>> OpenBLAS that was pulled in:
>> > >>>
>> > >>> created Make.user:
>> > >>> override USE_SYSTEM_BLAS = 1
>> > >>> override USE_BLAS64 = 0
>> > >>> override LIBBLAS = -L/opt/atlas/lib -ltatlas
>> > >>> override LIBBLASNAME = libtatlas
>> > >>>
>> > >>> I get a clean build all the way till the end and then this happens:
>> > >>>
>> > >>> ....
>> > >>> JULIA usr/lib/julia/inference.ji
>> > >>> /bin/sh: line 1: 6967 Segmentation fault
>> /localhome/geert/src/julia/usr/bin/julia -C native --output-ji
>> /localhome/geert/src/julia/usr/lib/julia/inference.ji --startup-file=no
>> coreimg.jl
>> > >>> Makefile:215: recipe for target
>> > >>> '/localhome/geert/src/julia/usr/lib/julia/inference.ji'
>> failed
>> > >>> make[1]: *** [/localhome/geert/src/julia/usr/lib/julia/inference.ji]
>> Error 139
>> > >>> Makefile:96: recipe for target 'julia-inference' failed
>> > >>> make: *** [julia-inference] Error 2
>> > >>>
>> > >>> The machine and OS details are:
>> > >>>
>> > >>> geert@tulgpu505:~/src/julia$ lscpu
>> > >>> Architecture: ppc64le
>> > >>> Byte Order: Little Endian
>> > >>> CPU(s): 184
>> > >>> On-line CPU(s) list: 0-151,160-191
>> > >>> Thread(s) per core: 8
>> > >>> Core(s) per socket: 5
>> > >>> Socket(s): 4
>> > >>> NUMA node(s): 4
>> > >>> Model: 8247-42L
>> > >>> CPU max MHz: 3923.0000
>> > >>> CPU min MHz: 2061.0000
>> > >>> L1d cache: 64K
>> > >>> L1i cache: 32K
>> > >>> L2 cache: 512K
>> > >>> L3 cache: 8192K
>> > >>> NUMA node0 CPU(s): 0-47
>> > >>> NUMA node1 CPU(s): 48-95
>> > >>> NUMA node16 CPU(s): 96-143
>> > >>> NUMA node17 CPU(s): 144-151,160-191
>> > >>> geert@tulgpu505:~/src/julia$ cat /etc/os-release
>> > >>> NAME="Ubuntu"
>> > >>> VERSION="14.10 (Utopic Unicorn)"
>> > >>> ID=ubuntu
>> > >>> ID_LIKE=debian
>> > >>> PRETTY_NAME="Ubuntu 14.10"
>> > >>> VERSION_ID="14.10"
>> > >>> HOME_URL="http://www.ubuntu.com/";
>> > >>> SUPPORT_URL="http://help.ubuntu.com/";
>> > >>> BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/";
>> > >>> geert@tulgpu505:~/src/julia$ uname -a
>> > >>> Linux tulgpu505 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12
>> 22:07:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux
>> > >>> geert@tulgpu505:~/src/julia$
>> > >>>
>> > >>> Any suggestions?
>> > >>>
>> > >>> Thanks,
>> > >>>
>> > >>> Geert
>> > >>
>> > >
>> >
>>
>>
>
mlp_iris.jl
Description: Binary data
