Hey Jack,
On 21 November 2016 at 00:15, Jack Perdue
<[email protected]<mailto:[email protected]>> wrote:
1) --use-existing-modules did not do what I expected
(I was curious why it was there). Although I had
had all dependencies recompiled (with modules) using
just their GCC subtoolchain (this was testing with foss),
if I use that option, EB insisted using the foss version
dependencies instead of the GCC version. Not what
I expected. Removing it got modules with min toolchains.
Yeah, it starts searching from the top when you use this...so if you have a
version installed with foss it will pick that over GCC.
load("libpng/1.6.24-iccifort-2016.3.210-GCC-5.4.0-2.26")
load("bzip2/1.0.6-iccifort-2016.3.210-GCC-5.4.0-2.26")
load("zlib/1.2.8-GCCcore-5.4.0")
I would've preferred it use zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26
This was exactly the use case for having `--use-existing-modules`, but the idea
only works if you're not starting from a module tree that already has a fully
populated top level (since it will always stop searching there). In other
words, if I start from a clean slate and explicitly install
zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26 (because I know Intel compiled
zlib gives better performance since it has been explicitly tuned) and then
build freetype with minimal-toolchains and use-existing-modules, it will chose
zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26 over the one in GCCcore when
appropriate.
Alan
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected]<mailto:[email protected]> http://hprc.tamu.edu
HPRC Helpdesk: [email protected]<mailto:[email protected]>
On 11/19/2016 01:18 PM, Kenneth Hoste wrote:
Hi Jack,
On 19/11/16 18:42, Jack Perdue wrote:
Ah... nevermind... it has to be in the ROBOTPATH,
Enabling --minimal-toolchains means that EasyBuild will go and hunt for
easyconfig files that provide dependencies with subtoolchains like iccifort
first, before it considers the full ('intel', in this case) toolchai.
So, yes, the robot search path is important here. Easyconfig files that are not
available there will not be considered.
Check with --dry-run or -D to see which easyconfigs 'eb' will pick up.
not just existing in
$EASYBUILD_INSTALLPATH_SOFTWARE/<name>/<version>/easybuild/<name>-<version>.eb
The easyconfig file that was used is indeed copied to the install directory,
but EasyBuild doesn't pick up on it later (unless you include each
<prefix>/</name>/<version>/easybuild/ directory in your robot search path).
Some people do include the easyconfigs archive (where easyconfigs are also
being copied to) in their robot search path, so they can easily reinstall
something they've installed before, or to let the robot pick up on it later.
where EasyBuild often looks (or at least used to).
I don't think EasyBuild ever picked up the easyconfig file that was copied to
the install directory automagically...
That gets me further... then it fails with:
These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26
== 2016-11-19 11:35:19,503 build_log.py:220 INFO FAILED: Installation ended
unsuccessfully (build directory: /tmp/easybuild/freetype/2.6.5/intel-2016b):
build failed (first 300 chars): Sanity check failed: loading fake module
failed: 'Changing environment as dictated by module failed: name \'false\' is
not defined (stdout: \nfalse\n, stderr: Lmod has detected the following error:
These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26"\n\n
== 2016-11-19 11:35:19,504 build_log.py:149 ERROR EasyBuild crashed with an
error (at ?:124 in __init__): build failed (first 300 chars): Sanity check
failed: loading fake module failed: 'Changing environment as dictated by module
failed: name \'false\' is not defined (stdout: \nfalse\n, stderr: Lmod has
detected the following error: These module(s) exist but cannot be loaded as
requested: "iccifort/2016.3.210-GCC-5.4.0-2.26"\n\n Tr (at
easybuild/main.py:118 in build_and_install_software)
== 2016-11-19 11:35:19,505 build_log.py:149 ERROR EasyBuild crashed with an
error (at ?:124 in __init__): Build of
/general/sw/eb/hprc/ec-dev/terra/freetype-2.6.5-intel-2016b.eb failed (err:
'build failed (first 300 chars): Sanity check failed: loading fake module
failed: \'Changing environment as dictated by module failed: name \\\'false\\\'
is not defined (stdout: \\nfalse\\n, stderr: Lmod has detected the following
error: These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26"\\n\\n Tr') (at easybuild/main.py:150 in
build_and_install_software)
This looks like a problem with your current $MODULEPATH.
It seems like you're installing to a different location than usual to test, but
remember that EasyBuild will (only) include the '/modules/all' subdirectory of
the location you're installing to to $MODULEPATH. If you also have modules in
other locations that it should be picked up on, do a 'module use' on those
locations first.
I hope this helps,
Kenneth
jack
On 11/19/2016 11:18 AM, Jack Perdue wrote:
Howdy again,
Alright I've tried the environment and the command line
and not having any luck. For environment, I have:
pushenv("EASYBUILD_MINIMAL_TOOLCHAINS", "1")
pushenv("EASYBUILD_USE_EXISTING_MODULES", "1")
I also built:
bzip2/1.0.6-iccifort-2016.3.210-GCC-5.4.0-2.26
libpng/1.6.24-iccifort-2016.3.210-GCC-5.4.0-2.26
zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26
I force a rebuild of the provided freetype-2.6.5-intel-2016b.eb
(which has those as dependencies) with the above settings
enabled (and/or on the command line), and I still get a freetype
that uses bzip2/1.0.6-intel-2016b (and not the one above).
intel/2016b is based on iccifort/2016.3.210-GCC-5.4.0-2.26.
Why doesn't it use the minimal toolchain version?
What am I missing?
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected]<mailto:[email protected]> http://hprc.tamu.edu
HPRC Helpdesk: [email protected]<mailto:[email protected]>
On 11/19/2016 01:39 AM, Kenneth Hoste wrote:
Hi Jack,
The rule is the same for all configuration options: strip off the --, all caps,
- becomes _, prefix with EASYBUILD_
So, you need to define $EASYBUILD_MINIMAL_TOOLCHAINS .
Also, make sure you read the documentation at
http://easybuild.readthedocs.io/en/latest/Manipulating_dependencies.html#using-minimal-toolchains-for-dependencies
<https://urldefense.proofpoint.com/v2/url?u=http-3A__easybuild.readthedocs.io_en_latest_Manipulating-5Fdependencies.html-23using-2Dminimal-2Dtoolchains-2Dfor-2Ddependencies&d=CwMFAg&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=lMiSfU2GikqWdQbwozX3Y7KXPniN6Hlo0pz_PSaStU8&s=TjtxIb9vxJnsiX2yDsMKwv4zEnsuhJwRUHTUb2r4U1c&e=>,
since there are additional config options you should consider that relate to
--minimal-toolchains.
regard,
Kenneth
Op 19 nov. 2016 om 07:44 heeft Jack Perdue
<[email protected]<mailto:[email protected]>
<mailto:[email protected]<mailto:[email protected]>>> het volgende geschreven:
Howdy EB gods,
What's the environment variable for --minimal-toolchain
(e.g. EASYBUILD_MINIMAL_TOOLCHAIN=1 ??)
http://easybuild.readthedocs.io/en/latest/Manipulating_dependencies.html#using-minimal-toolchains-for-dependencies
<https://urldefense.proofpoint.com/v2/url?u=http-3A__easybuild.readthedocs.io_en_latest_Manipulating-5Fdependencies.html-23using-2Dminimal-2Dtoolchains-2Dfor-2Ddependencies&d=CwMFAg&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=lMiSfU2GikqWdQbwozX3Y7KXPniN6Hlo0pz_PSaStU8&s=TjtxIb9vxJnsiX2yDsMKwv4zEnsuhJwRUHTUb2r4U1c&e=>
I'd like it as a default. jack
--
Dr. Alan O'Cais
E-CAM Software Manager
Juelich Supercomputing Centre
Forschungszentrum Juelich GmbH
52425 Juelich, Germany
Phone: +49 2461 61 5213
Fax: +49 2461 61 6656
E-mail: [email protected]<mailto:[email protected]>
WWW: http://www.fz-juelich.de/ias/jsc/EN
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Forschungszentrum Juelich GmbH
52425 Juelich
Sitz der Gesellschaft: Juelich
Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
Prof. Dr. Sebastian M. Schmidt
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
