Howdy Ken/all,
In the end, I ended up getting to work after dealing
with the following two issues:
1) --use-existing-modules did not do what I expected
(I was curious why it was there). Although I had
had all dependencies recompiled (with modules) using
just their GCC subtoolchain (this was testing with foss),
if I use that option, EB insisted using the foss version
dependencies instead of the GCC version. Not what
I expected. Removing it got modules with min toolchains.
2) There is something going on with Lmod7 and the sanity
check when using minimal-toolchains. I can rollback to
6.6 and have no problems but 7+ fails.
A couple quick notes on the docs (ReadTheDocs):
1) Python.eb is a horrible example since it takes forever
to demonstrate what's going on. freetype has been
a good test case.
2) There is no GCC/4.9.3 subtoollchain for foss/2015a.
(its 4.9.3-2.25 last time I checked).
One final note, I guess I need to hide some of the
packages used to create toolchains. In the end,
(back to the intel test case), I ended up with:
load("libpng/1.6.24-iccifort-2016.3.210-GCC-5.4.0-2.26")
load("bzip2/1.0.6-iccifort-2016.3.210-GCC-5.4.0-2.26")
load("zlib/1.2.8-GCCcore-5.4.0")
I would've preferred it use zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected] http://hprc.tamu.edu
HPRC Helpdesk: [email protected]
On 11/19/2016 01:18 PM, Kenneth Hoste wrote:
Hi Jack,
On 19/11/16 18:42, Jack Perdue wrote:
Ah... nevermind... it has to be in the ROBOTPATH,
Enabling --minimal-toolchains means that EasyBuild will go and hunt
for easyconfig files that provide dependencies with subtoolchains like
iccifort first, before it considers the full ('intel', in this case)
toolchai.
So, yes, the robot search path is important here. Easyconfig files
that are not available there will not be considered.
Check with --dry-run or -D to see which easyconfigs 'eb' will pick up.
not just existing in
$EASYBUILD_INSTALLPATH_SOFTWARE/<name>/<version>/easybuild/<name>-<version>.eb
The easyconfig file that was used is indeed copied to the install
directory, but EasyBuild doesn't pick up on it later (unless you
include each <prefix>/</name>/<version>/easybuild/ directory in your
robot search path).
Some people do include the easyconfigs archive (where easyconfigs are
also being copied to) in their robot search path, so they can easily
reinstall something they've installed before, or to let the robot pick
up on it later.
where EasyBuild often looks (or at least used to).
I don't think EasyBuild ever picked up the easyconfig file that was
copied to the install directory automagically...
That gets me further... then it fails with:
These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26
== 2016-11-19 11:35:19,503 build_log.py:220 INFO FAILED: Installation
ended unsuccessfully (build directory:
/tmp/easybuild/freetype/2.6.5/intel-2016b): build failed (first 300
chars): Sanity check failed: loading fake module failed: 'Changing
environment as dictated by module failed: name \'false\' is not
defined (stdout: \nfalse\n, stderr: Lmod has detected the following
error: These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26"\n\n
== 2016-11-19 11:35:19,504 build_log.py:149 ERROR EasyBuild crashed
with an error (at ?:124 in __init__): build failed (first 300 chars):
Sanity check failed: loading fake module failed: 'Changing
environment as dictated by module failed: name \'false\' is not
defined (stdout: \nfalse\n, stderr: Lmod has detected the following
error: These module(s) exist but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26"\n\n Tr (at
easybuild/main.py:118 in build_and_install_software)
== 2016-11-19 11:35:19,505 build_log.py:149 ERROR EasyBuild crashed
with an error (at ?:124 in __init__): Build of
/general/sw/eb/hprc/ec-dev/terra/freetype-2.6.5-intel-2016b.eb failed
(err: 'build failed (first 300 chars): Sanity check failed: loading
fake module failed: \'Changing environment as dictated by module
failed: name \\\'false\\\' is not defined (stdout: \\nfalse\\n,
stderr: Lmod has detected the following error: These module(s) exist
but cannot be loaded as requested:
"iccifort/2016.3.210-GCC-5.4.0-2.26"\\n\\n Tr') (at
easybuild/main.py:150 in build_and_install_software)
This looks like a problem with your current $MODULEPATH.
It seems like you're installing to a different location than usual to
test, but remember that EasyBuild will (only) include the
'/modules/all' subdirectory of the location you're installing to to
$MODULEPATH. If you also have modules in other locations that it
should be picked up on, do a 'module use' on those locations first.
I hope this helps,
Kenneth
jack
On 11/19/2016 11:18 AM, Jack Perdue wrote:
Howdy again,
Alright I've tried the environment and the command line
and not having any luck. For environment, I have:
pushenv("EASYBUILD_MINIMAL_TOOLCHAINS", "1")
pushenv("EASYBUILD_USE_EXISTING_MODULES", "1")
I also built:
bzip2/1.0.6-iccifort-2016.3.210-GCC-5.4.0-2.26
libpng/1.6.24-iccifort-2016.3.210-GCC-5.4.0-2.26
zlib/1.2.8-iccifort-2016.3.210-GCC-5.4.0-2.26
I force a rebuild of the provided freetype-2.6.5-intel-2016b.eb
(which has those as dependencies) with the above settings
enabled (and/or on the command line), and I still get a freetype
that uses bzip2/1.0.6-intel-2016b (and not the one above).
intel/2016b is based on iccifort/2016.3.210-GCC-5.4.0-2.26.
Why doesn't it use the minimal toolchain version?
What am I missing?
Jack Perdue
Lead Systems Administrator
High Performance Research Computing
TAMU Division of Research
[email protected] http://hprc.tamu.edu
HPRC Helpdesk: [email protected]
On 11/19/2016 01:39 AM, Kenneth Hoste wrote:
Hi Jack,
The rule is the same for all configuration options: strip off the
--, all caps, - becomes _, prefix with EASYBUILD_
So, you need to define $EASYBUILD_MINIMAL_TOOLCHAINS .
Also, make sure you read the documentation at
http://easybuild.readthedocs.io/en/latest/Manipulating_dependencies.html#using-minimal-toolchains-for-dependencies
<https://urldefense.proofpoint.com/v2/url?u=http-3A__easybuild.readthedocs.io_en_latest_Manipulating-5Fdependencies.html-23using-2Dminimal-2Dtoolchains-2Dfor-2Ddependencies&d=CwMFAg&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=lMiSfU2GikqWdQbwozX3Y7KXPniN6Hlo0pz_PSaStU8&s=TjtxIb9vxJnsiX2yDsMKwv4zEnsuhJwRUHTUb2r4U1c&e=>,
since there are additional config options you should consider that
relate to --minimal-toolchains.
regard,
Kenneth
Op 19 nov. 2016 om 07:44 heeft Jack Perdue <[email protected]
<mailto:[email protected]>> het volgende geschreven:
Howdy EB gods,
What's the environment variable for --minimal-toolchain
(e.g. EASYBUILD_MINIMAL_TOOLCHAIN=1 ??)
http://easybuild.readthedocs.io/en/latest/Manipulating_dependencies.html#using-minimal-toolchains-for-dependencies
<https://urldefense.proofpoint.com/v2/url?u=http-3A__easybuild.readthedocs.io_en_latest_Manipulating-5Fdependencies.html-23using-2Dminimal-2Dtoolchains-2Dfor-2Ddependencies&d=CwMFAg&c=ODFT-G5SujMiGrKuoJJjVg&r=yuyoBkmTkIQPbv1BTF9U27ww5Lm7GhsMmWcQG9gmjbA&m=lMiSfU2GikqWdQbwozX3Y7KXPniN6Hlo0pz_PSaStU8&s=TjtxIb9vxJnsiX2yDsMKwv4zEnsuhJwRUHTUb2r4U1c&e=>
I'd like it as a default. jack
