Dear Niek,
Sorry that you're hitting these issues, let's try and get them sorted...
On 18/11/2019 20:00, Niek de Klein wrote:
Hi all,
I am trying to have my own installation of EasyBuild, but I have 3 problems:
1. When I try to run bootstrap.py it gives an error that md5sums are
missing, but I'm not sure where to add them as I thought that should
then have been included in the EasyBuild-4.0.1.eb file:
easybuild.tools.build_log.EasyBuildError: "Build of
/tmp/tmpfcUSAH/EasyBuild-4.0.1.eb failed (err: 'build failed (first 300
chars): Missing checksum for easybuild-framework-4.0.1.tar.gz')"
Sorry for not responding to the GitHub issue [1] you opened on this
(will do so now with the answer below).
We still need to clean up the bootstrap script a bit after the EasyBuild
v4.0 release, it's still doing a bunch of stuff it shouldn't anymore,
and it's not compatible yet with Python 3 (see also
https://github.com/easybuilders/easybuild-framework/pull/3028).
Your active EasyBuild configuration has --enforce-checksums enabled (via
$EASYBUILD_ENFORCE_CHECKSUMS), which explains the problem you're seeing.
The bootstrap script auto-generates an easyconfig file, but it doesn't
include checksums (it could I guess, with a bit more work).
You can dance around this though, by temporarily disabling the checksums
requirement, only for the bootstrap:
EASYBUILD_DISABLE_ENFORCE_CHECKSUMS=1 python bootstrap <path>
If you want to, you can bootstrap EasyBuild to a temporary location (for
example /tmp/$USER), and then use that installation to pull in an
easyconfig file for EasyBuild v4.0.1 that does have checksums, using "eb
--from-pr 9149 --installpath <final_path>" (which pulls in the
easyconfig from
https://github.com/easybuilders/easybuild-easyconfigs/pull/9149/files)
for the actual installation you intend to use.
2. I changed the robo and installation path to be different, but I keep
getting "Failed to determine minimal toolchain for dep ..." errors. The
eb --show-config output is at the end of this e-mail, and shows that my
robot-paths is
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/,
which is where I cloned the easyconfigs repository.
What happens if you use "eb --search" with this setup, can 'eb' find
easyconfig files?
Try something like "eb --search tensorflow-1.14" for example.
3. Even if the robo path wasn't working, tools that are already in my
module avail are not being picked up. For example, EasyBuild that is not
working uses "toolchain = {'name': 'foss', 'version': '2018b'}" and has
as one of the dependencies "('ncurses', '6.0'),". When I try to install
it I get
ERROR: Failed to process easyconfig
/groups/umcg-biogen/tmp01/tools/brain_eQTL/EasyBuild/easyconfigs/SAMtools-1.5-foss-2018b.eb:
Failed to determine minimal toolchain for dep ncurses 6.0
but when I do ml avail ncur I get:
------------------------------------------------------
/groups/umcg-biogen/tmp01/apps/modules/devel
-------------------------------------------------------
ncurses/6.0-foss-2018b
-------------------------------------------------------------------
/apps/modules/devel
-------------------------------------------------------------------
ncurses/6.0 ncurses/6.1-foss-2018b ncurses/6.1-GCCcore-7.3.0 (D)
So the ncurses module is already available
Am I missing something in my configuration?
This is a consequence of having --minimal-toolchains enabled, which
requires that EasyBuild is able to find the easyconfig file for the
toolchain (foss-2018b.eb in this case).
It needs to figure out what the (versions of) the subtoolchains are for
foss/2018b, so it can check for ncurses installed with GCCcore first,
before gompi, before foss.
So this is basically the same issue as with problem 2, I bet "eb
--search ^foss-2018b.eb" doesn't give you any result?
Are you 100% sure the path you specified for --robot-paths is correct?
Note that you can also prepend or append to the robot search path
(rather than override the default), see
https://easybuild.readthedocs.io/en/latest/Using_the_EasyBuild_command_line.html#controlling-robot-search-path
.
I hope this helps...
regards,
Kenneth
[1] https://github.com/easybuilders/easybuild-framework/issues/3079
Thanks for your help!
Niek de Klein
#
# Current EasyBuild configuration
# (C: command line argument, D: default value, E: environment variable,
F: configuration file)
#
buildpath (E) =
/groups/umcg-biogen/tmp01/apps//.tmp/easybuild/builds/
containerpath (D) =
/home/umcg-ndeklein/.local/easybuild/containers
enforce-checksums (E) = True
installpath (E) = /groups/umcg-biogen/tmp01/apps/
minimal-toolchains (E) = True
repositorypath (D) =
/home/umcg-ndeklein/.local/easybuild/ebfiles_repo
robot-paths (E) =
/groups/umcg-biogen/tmp01/apps/easybuild-easyconfigs/easybuild/easyconfigs/
set-gid-bit (E) = True
sourcepath (E) = /groups/umcg-biogen/tmp01/apps//sources/
umask (E) = 002
verify-easyconfig-filenames (E) = True
