Re: [julia-users] Pre-compiling images for different CPUs on GCloud
An even better command is:
```
julia -e 'println(Sys.cpu_name)'
```
-erik
On Sat, Feb 6, 2016 at 7:24 PM, Pavel wrote:
> The "core-avx2" target worked (LLVL 3.4), and it was in the output of `llc
> --version`. I was able to precompile locally and deploy on GCloud haswell.
> The run time of my test suite was effectively the same with that and with
> the "native" option precompilation. Thank you Milan and Erik for your input.
>
> On Saturday, February 6, 2016 at 7:09:17 AM UTC-8, Erik Schnetter wrote:
>>
>> You can run `llc --version` to determine the host's CPU type. You
>> should probably use the same llc that comes with the LLVM that Julia
>> is using.
>>
>> The exact command is
>>
>> `llc --version | awk '/Host CPU:/ { print $3; }')`
>>
>> -erik
>>
>>
>> On Sat, Feb 6, 2016 at 9:50 AM, Milan Bouchet-Valat
>> wrote:
>> > Le vendredi 05 février 2016 à 18:24 -0800, Pavel a écrit :
>> >> I tried this approach using a local machine with
>> >> product: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
>> >> which in my understanding belongs to the "haswell" type.
>> >>
>> >> setting
>> >> build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
>> >> "haswell", "/home/juser/jimg.jl", force=true)
>> >>
>> >> resulted in
>> >> 'haswell' is not a recognized processor for this target (ignoring
>> >> processor)
>> > Have you looked at the output of llc -mattr=help (or usr/bin/llc
>> > -mattr=help from the Julia install/build directory)? If this is with
>> > LLVM 3.3, then it doesn't look like it supported "haswell" as a target.
>> > You should be able to specify "core-avx2" instead.
>> >
>> >> On the other hand, setting "native" instead, after deploying on
>> >> GCloud haswell-zone says that the image was precompiled for a
>> >> different architecture.
>> > By definition, "native" isn't intended to be portable, so it doesn't
>> > sound unexpected.
>> >
>> >
>> > Regards
>> >
>> >
>> >> On Friday, February 5, 2016 at 1:01:37 AM UTC-8, Milan Bouchet-Valat
>> >> wrote:
>> >> > Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
>> >> > > I am deploying a CPU-intensive application on Google Cloud
>> >> > Compute
>> >> > > with Julia code running inside a Docker container. GCloud
>> >> > instances
>> >> > > have a few different CPU architectures depending on the zone.
>> >> > > Ideally, I would like to pre-compile Julia images (i.e.
>> >> > different
>> >> > > docker container images) for all of those architectures.
>> >> > >
>> >> > > Currently the following runs on container launch:
>> >> > >
>> >> > > include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
>> >> > > "build_sysimg.jl"))
>> >> > >
>> >> > build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
>> >> > > "native", "/home/juser/jimg.jl", force=true)
>> >> > >
>> >> > > so that the "native" setting pre-compiles for the CPU that these
>> >> > > commands are running on. That takes time and having that done at
>> >> > the
>> >> > > container image build time would save cloud runtime.
>> >> > >
>> >> > > Would it be possible to pre-compile for different CPUs on my
>> >> > > development machine or do I need to run the pre-compilation on
>> >> > those
>> >> > > exact CPUs? If the former is possible, what build_sysimg
>> >> > arguments
>> >> > > would help? GCloud virtual machine architectures of interest
>> >> > include
>> >> > > Sandy Bridge, Ivy Bridge, and Haswell CPU types.
>> >> > You should be able to build images for older architectures on a
>> >> > machine
>> >> > with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge
>> >> > on
>> >> > a Haswell machine. I don't think it would work in the other
>> >> > direction,
>> >> > since some instructions wouldn't be supported and the code
>> >> > wouldn't
>> >> > run.
>> >> >
>> >> > So in your case you could replace "native" with "sandybridge",
>> >> > "ivybridge", "haswell", or any value from the list printed by
>> >> > llc -mattr=help.
>> >> >
>> >> >
>> >> > Regards
>>
>>
>>
>> --
>> Erik Schnetter
>> http://www.perimeterinstitute.ca/personal/eschnetter/
--
Erik Schnetter
http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Pre-compiling images for different CPUs on GCloud
The "core-avx2" target worked (LLVL 3.4), and it was in the output of `llc
--version`. I was able to precompile locally and deploy on GCloud haswell.
The run time of my test suite was effectively the same with that and with
the "native" option precompilation. Thank you Milan and Erik for your input.
On Saturday, February 6, 2016 at 7:09:17 AM UTC-8, Erik Schnetter wrote:
>
> You can run `llc --version` to determine the host's CPU type. You
> should probably use the same llc that comes with the LLVM that Julia
> is using.
>
> The exact command is
>
> `llc --version | awk '/Host CPU:/ { print $3; }')`
>
> -erik
>
>
> On Sat, Feb 6, 2016 at 9:50 AM, Milan Bouchet-Valat > wrote:
> > Le vendredi 05 février 2016 à 18:24 -0800, Pavel a écrit :
> >> I tried this approach using a local machine with
> >> product: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
> >> which in my understanding belongs to the "haswell" type.
> >>
> >> setting
> >> build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> >> "haswell", "/home/juser/jimg.jl", force=true)
> >>
> >> resulted in
> >> 'haswell' is not a recognized processor for this target (ignoring
> >> processor)
> > Have you looked at the output of llc -mattr=help (or usr/bin/llc
> > -mattr=help from the Julia install/build directory)? If this is with
> > LLVM 3.3, then it doesn't look like it supported "haswell" as a target.
> > You should be able to specify "core-avx2" instead.
> >
> >> On the other hand, setting "native" instead, after deploying on
> >> GCloud haswell-zone says that the image was precompiled for a
> >> different architecture.
> > By definition, "native" isn't intended to be portable, so it doesn't
> > sound unexpected.
> >
> >
> > Regards
> >
> >
> >> On Friday, February 5, 2016 at 1:01:37 AM UTC-8, Milan Bouchet-Valat
> >> wrote:
> >> > Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
> >> > > I am deploying a CPU-intensive application on Google Cloud
> >> > Compute
> >> > > with Julia code running inside a Docker container. GCloud
> >> > instances
> >> > > have a few different CPU architectures depending on the zone.
> >> > > Ideally, I would like to pre-compile Julia images (i.e.
> >> > different
> >> > > docker container images) for all of those architectures.
> >> > >
> >> > > Currently the following runs on container launch:
> >> > >
> >> > > include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
> >> > > "build_sysimg.jl"))
> >> > >
> >> > build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> >> > > "native", "/home/juser/jimg.jl", force=true)
> >> > >
> >> > > so that the "native" setting pre-compiles for the CPU that these
> >> > > commands are running on. That takes time and having that done at
> >> > the
> >> > > container image build time would save cloud runtime.
> >> > >
> >> > > Would it be possible to pre-compile for different CPUs on my
> >> > > development machine or do I need to run the pre-compilation on
> >> > those
> >> > > exact CPUs? If the former is possible, what build_sysimg
> >> > arguments
> >> > > would help? GCloud virtual machine architectures of interest
> >> > include
> >> > > Sandy Bridge, Ivy Bridge, and Haswell CPU types.
> >> > You should be able to build images for older architectures on a
> >> > machine
> >> > with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge
> >> > on
> >> > a Haswell machine. I don't think it would work in the other
> >> > direction,
> >> > since some instructions wouldn't be supported and the code
> >> > wouldn't
> >> > run.
> >> >
> >> > So in your case you could replace "native" with "sandybridge",
> >> > "ivybridge", "haswell", or any value from the list printed by
> >> > llc -mattr=help.
> >> >
> >> >
> >> > Regards
>
>
>
> --
> Erik Schnetter >
> http://www.perimeterinstitute.ca/personal/eschnetter/
>
Re: [julia-users] Pre-compiling images for different CPUs on GCloud
You can run `llc --version` to determine the host's CPU type. You
should probably use the same llc that comes with the LLVM that Julia
is using.
The exact command is
`llc --version | awk '/Host CPU:/ { print $3; }')`
-erik
On Sat, Feb 6, 2016 at 9:50 AM, Milan Bouchet-Valat wrote:
> Le vendredi 05 février 2016 à 18:24 -0800, Pavel a écrit :
>> I tried this approach using a local machine with
>> product: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
>> which in my understanding belongs to the "haswell" type.
>>
>> setting
>> build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
>> "haswell", "/home/juser/jimg.jl", force=true)
>>
>> resulted in
>> 'haswell' is not a recognized processor for this target (ignoring
>> processor)
> Have you looked at the output of llc -mattr=help (or usr/bin/llc
> -mattr=help from the Julia install/build directory)? If this is with
> LLVM 3.3, then it doesn't look like it supported "haswell" as a target.
> You should be able to specify "core-avx2" instead.
>
>> On the other hand, setting "native" instead, after deploying on
>> GCloud haswell-zone says that the image was precompiled for a
>> different architecture.
> By definition, "native" isn't intended to be portable, so it doesn't
> sound unexpected.
>
>
> Regards
>
>
>> On Friday, February 5, 2016 at 1:01:37 AM UTC-8, Milan Bouchet-Valat
>> wrote:
>> > Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
>> > > I am deploying a CPU-intensive application on Google Cloud
>> > Compute
>> > > with Julia code running inside a Docker container. GCloud
>> > instances
>> > > have a few different CPU architectures depending on the zone.
>> > > Ideally, I would like to pre-compile Julia images (i.e.
>> > different
>> > > docker container images) for all of those architectures.
>> > >
>> > > Currently the following runs on container launch:
>> > >
>> > > include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
>> > > "build_sysimg.jl"))
>> > >
>> > build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
>> > > "native", "/home/juser/jimg.jl", force=true)
>> > >
>> > > so that the "native" setting pre-compiles for the CPU that these
>> > > commands are running on. That takes time and having that done at
>> > the
>> > > container image build time would save cloud runtime.
>> > >
>> > > Would it be possible to pre-compile for different CPUs on my
>> > > development machine or do I need to run the pre-compilation on
>> > those
>> > > exact CPUs? If the former is possible, what build_sysimg
>> > arguments
>> > > would help? GCloud virtual machine architectures of interest
>> > include
>> > > Sandy Bridge, Ivy Bridge, and Haswell CPU types.
>> > You should be able to build images for older architectures on a
>> > machine
>> > with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge
>> > on
>> > a Haswell machine. I don't think it would work in the other
>> > direction,
>> > since some instructions wouldn't be supported and the code
>> > wouldn't
>> > run.
>> >
>> > So in your case you could replace "native" with "sandybridge",
>> > "ivybridge", "haswell", or any value from the list printed by
>> > llc -mattr=help.
>> >
>> >
>> > Regards
--
Erik Schnetter
http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Pre-compiling images for different CPUs on GCloud
Le vendredi 05 février 2016 à 18:24 -0800, Pavel a écrit :
> I tried this approach using a local machine with
> product: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
> which in my understanding belongs to the "haswell" type.
>
> setting
> build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> "haswell", "/home/juser/jimg.jl", force=true)
>
> resulted in
> 'haswell' is not a recognized processor for this target (ignoring
> processor)
Have you looked at the output of llc -mattr=help (or usr/bin/llc
-mattr=help from the Julia install/build directory)? If this is with
LLVM 3.3, then it doesn't look like it supported "haswell" as a target.
You should be able to specify "core-avx2" instead.
> On the other hand, setting "native" instead, after deploying on
> GCloud haswell-zone says that the image was precompiled for a
> different architecture.
By definition, "native" isn't intended to be portable, so it doesn't
sound unexpected.
Regards
> On Friday, February 5, 2016 at 1:01:37 AM UTC-8, Milan Bouchet-Valat
> wrote:
> > Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
> > > I am deploying a CPU-intensive application on Google Cloud
> > Compute
> > > with Julia code running inside a Docker container. GCloud
> > instances
> > > have a few different CPU architectures depending on the zone.
> > > Ideally, I would like to pre-compile Julia images (i.e.
> > different
> > > docker container images) for all of those architectures.
> > >
> > > Currently the following runs on container launch:
> > >
> > > include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
> > > "build_sysimg.jl"))
> > >
> > build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> > > "native", "/home/juser/jimg.jl", force=true)
> > >
> > > so that the "native" setting pre-compiles for the CPU that these
> > > commands are running on. That takes time and having that done at
> > the
> > > container image build time would save cloud runtime.
> > >
> > > Would it be possible to pre-compile for different CPUs on my
> > > development machine or do I need to run the pre-compilation on
> > those
> > > exact CPUs? If the former is possible, what build_sysimg
> > arguments
> > > would help? GCloud virtual machine architectures of interest
> > include
> > > Sandy Bridge, Ivy Bridge, and Haswell CPU types.
> > You should be able to build images for older architectures on a
> > machine
> > with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge
> > on
> > a Haswell machine. I don't think it would work in the other
> > direction,
> > since some instructions wouldn't be supported and the code
> > wouldn't
> > run.
> >
> > So in your case you could replace "native" with "sandybridge",
> > "ivybridge", "haswell", or any value from the list printed by
> > llc -mattr=help.
> >
> >
> > Regards
Re: [julia-users] Pre-compiling images for different CPUs on GCloud
I tried this approach using a local machine with
product: Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz
which in my understanding belongs to the "haswell" type.
setting
build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
"haswell", "/home/juser/jimg.jl", force=true)
resulted in
'haswell' is not a recognized processor for this target (ignoring
processor)
On the other hand, setting "native" instead, after deploying on GCloud
haswell-zone says that the image was precompiled for a different
architecture.
On Friday, February 5, 2016 at 1:01:37 AM UTC-8, Milan Bouchet-Valat wrote:
>
> Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
> > I am deploying a CPU-intensive application on Google Cloud Compute
> > with Julia code running inside a Docker container. GCloud instances
> > have a few different CPU architectures depending on the zone.
> > Ideally, I would like to pre-compile Julia images (i.e. different
> > docker container images) for all of those architectures.
> >
> > Currently the following runs on container launch:
> >
> > include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
> > "build_sysimg.jl"))
> > build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> > "native", "/home/juser/jimg.jl", force=true)
> >
> > so that the "native" setting pre-compiles for the CPU that these
> > commands are running on. That takes time and having that done at the
> > container image build time would save cloud runtime.
> >
> > Would it be possible to pre-compile for different CPUs on my
> > development machine or do I need to run the pre-compilation on those
> > exact CPUs? If the former is possible, what build_sysimg arguments
> > would help? GCloud virtual machine architectures of interest include
> > Sandy Bridge, Ivy Bridge, and Haswell CPU types.
> You should be able to build images for older architectures on a machine
> with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge on
> a Haswell machine. I don't think it would work in the other direction,
> since some instructions wouldn't be supported and the code wouldn't
> run.
>
> So in your case you could replace "native" with "sandybridge",
> "ivybridge", "haswell", or any value from the list printed by
> llc -mattr=help.
>
>
> Regards
>
Re: [julia-users] Pre-compiling images for different CPUs on GCloud
Le jeudi 04 février 2016 à 17:17 -0800, Pavel a écrit :
> I am deploying a CPU-intensive application on Google Cloud Compute
> with Julia code running inside a Docker container. GCloud instances
> have a few different CPU architectures depending on the zone.
> Ideally, I would like to pre-compile Julia images (i.e. different
> docker container images) for all of those architectures.
>
> Currently the following runs on container launch:
>
> include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
> "build_sysimg.jl"))
> build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
> "native", "/home/juser/jimg.jl", force=true)
>
> so that the "native" setting pre-compiles for the CPU that these
> commands are running on. That takes time and having that done at the
> container image build time would save cloud runtime.
>
> Would it be possible to pre-compile for different CPUs on my
> development machine or do I need to run the pre-compilation on those
> exact CPUs? If the former is possible, what build_sysimg arguments
> would help? GCloud virtual machine architectures of interest include
> Sandy Bridge, Ivy Bridge, and Haswell CPU types.
You should be able to build images for older architectures on a machine
with a newer one, i.e. build images for Sandy Bridge and Ivy Bridge on
a Haswell machine. I don't think it would work in the other direction,
since some instructions wouldn't be supported and the code wouldn't
run.
So in your case you could replace "native" with "sandybridge",
"ivybridge", "haswell", or any value from the list printed by
llc -mattr=help.
Regards
[julia-users] Pre-compiling images for different CPUs on GCloud
I am deploying a CPU-intensive application on Google Cloud Compute with
Julia code running inside a Docker container. GCloud instances have a few
different CPU architectures depending on the zone. Ideally, I would like to
pre-compile Julia images (i.e. different docker container images) for all
of those architectures.
Currently the following runs on container launch:
include(joinpath(JULIA_HOME, Base.DATAROOTDIR, "julia",
"build_sysimg.jl"))
build_sysimg(joinpath(dirname(Libdl.dlpath("libjulia")),"sys"),
"native", "/home/juser/jimg.jl", force=true)
so that the "native" setting pre-compiles for the CPU that these commands
are running on. That takes time and having that done at the container image
build time would save cloud runtime.
Would it be possible to pre-compile for different CPUs on my development
machine or do I need to run the pre-compilation on those exact CPUs? If the
former is possible, what build_sysimg arguments would help? GCloud virtual
machine architectures of interest include Sandy Bridge, Ivy Bridge, and
Haswell CPU types.
Thanks for reading,
Pavel
