Dear EasyBuilders,
We're pleased to announce the release of EasyBuild v4.4.0 [1].
In my view, and I don't say this lightly, this is the best EasyBuild
release ever!
Not only because the version number has been increased, but also because:
- it includes initial support for custom toolchains using the Fujitsu
compiler & libraries, which are leveraged on Fugaku, the current fastest
(publicly known) supercomputer!
- easyconfig files for the 2021a update of the common toolchains (foss +
intel) are included;
- during the process of preparing this release, we merged the 10,000th
pull request in the easyconfigs GitHub repository!
- various small but yet useful enhancements have been made (more info
below);
- much to our surprise, we found a couple of bugs in previous EasyBuild
versions, which have been fixed (more info below);
GitHub Actions being flaky or even down for several hours multiple times
in the recent weeks, including today during the release process, didn't
prevent us from pushing out this release. Nice try Microsoft!
EasyBuild v4.4.0 is primarily a feature release, but also includes
several bug fixes & minor enhancements.
Highlights for this release are listed below. More details are available
in the release notes [2] which includes links to the respective pull
requests for more detailed information.
(this information is also available at
https://github.com/easybuilders/easybuild/releases/tag/easybuild-v4.4.0)
## Highlighted enhancements
[enhancements that (may) warrant updating existing installations are
marked with (***)]
- performance improvements for easyconfig parsing and eb startup;
- support for downloading easyconfigs from multiple PRs with --from-pr;
- support for re-running the sanity check for existing installations
(without making modifications to the installation), via "eb
--sanity-check-only";
- toolchain definition for Fujitsu toolchain for use on Fugaku;
- allow checking whether specific libraries are (not) linked into
installed binaries/libraries in sanity check (see various ways to
specify banned/required libraries);
- update_build_option function to update specific build options after
initializing the EasyBuild configuration;
- run post-install commands specified for a specific extension;
- add support for skipping the installation of extension via "eb
--skip-extensions";
- software-specific easyblocks for FlexiBLAS and dm-reverb;
- custom easyblock to install OpenSSL wrapper for OpenSSL installed in
OS, with fallback to build and install OpenSSL from source if not
available in OS;
- enable sanity_pip_check by default for Python easyconfigs if pip >=
9.0 will be installed;
- add IceLake detection to OpenBLAS 0.3.12 and 0.3.15;
## Prominent bug fixes & changes
[bug fixes or changes that (may) warrant reinstalling easyconfigs are
marked with (***)]
- re-enable write permissions when installing with read-only-installdir;
- avoid metadata greedy behavior when probing for external module
metadata (mostly relevant for integration with Cray Programming
Environment);
- also run sanity check for extensions when using --module-only (you can
use --skip-extensions to skip sanity checking of extensions when using
--module-only);
- fix use of --module-only on existing installations without write
permissions;
- use unload/load in ModuleGeneratorLua.swap_module, since swap is not
supported by Lmod;
- update HierarchicalMNS to also return ‘Toolchain/<name>/<version>’ as
$MODULEPATH extension for cpe* Cray toolchains;
- enhance sched_getaffinity function to avoid early crash when counting
available cores on systems with more than 1024 cores;
- (***) enhance Python easyblock: add option to install pip with core
Python, tweak defaults, create unversioned pip symlink;
- make custom easyblocks for GROMACS ad Tkinter work with --module-only;
- make sure that self.python_cmd is set before using it in
PythonPackage.sanity_check_step to make PythonBundle easyblock
comaptible with --module-only;
- (***) add patch to fix GCC 10.2.0 rejecting valid code on PPC;
- (***) update easyconfigs for binutils 2.35 to use binutils 2.35.2
source tarball instead to pick up bug fixes;
- (***) fix test failure in TensorFlow 2.4.1 on recent CUDA drivers;
- (***) add patch to fix hardcoded num_cores in DMCfun extension
included with R 4.0.x;
- (***) fix typo in Delly easyconfig to actually do parallel build;
- (***) fix potential memory leak in OpenBLAS 0.3.12;
- (***) add patches for PyTorch 1.7.1 avoiding failures on POWER and A100;
- fix source URLs for recent Boost and Boost.Python versions;
## Other changes
- tweak foss toolchain definition to switch from OpenBLAS to FlexiBLAS
in foss/2021a;
- don’t skip sanity check for --module-only --rebuild (sanity check is
still skipped with --module-only --force);
- consistently use pip to install Python packages in recent Python
easyconfigs;
- deprecate adding a non-existing path to $MODULEPATH via ModulesTool.use;
- drop support for running EasyBuild with Python 2.6;
## Supported software
Easyconfigs for 2021a version of custom toolchains:
foss/2021a:
GCC 10.3.0 + binutils 2.36.1
OpenMPI 4.1.1
FlexiBLAS 3.0.4 with OpenBLAS 0.3.15 as backend
ScaLAPACK 2.1.0
FFTW 3.3.9
intel/2021a:
GCC 10.3.0 + binutils 2.36.1 as base
intel-compilers 2021.2.0
impi 2021.2.0
imkl 2021.2.0
Support for installing 58 new software packages has been added,
including (but not limited to):
BirdNET
cell2location
cryoDRGN
FlexiBLAS
fsom
gemmi
georges
hifiasm
intervaltree
LDC
librosa
loompy
mongolite
NewHybrids
pagmo
pyfaidx
pyfasta
pygmo
R-INLA
rMATS-turbo
SeqLib
smithwaterman
SSW
TagDust
TALON
TranscriptClean
xESMF
=> This brings the total number of supported software packages to 2,349!
An up-to-date list of supported software is available at [4].
Various software updates have been added, including (but not limited to):
ABINIT 9.4.1
BLIS 0.8.1
Bonito 0.3.8
ccache 4.2.1
CLHEP 2.4.4.0
CMake 3.20.1
CUDA(core) 11.3.0
CuPy 8.5.0
Eigen 3.3.9
FFTW 3.3.9
GATK 4.2.0.0
GCC 10.3.0 + 11.1.0
GDRCopy 2.2
GLPK 5.0
GROMACS 2021.2
GTDB-Tk 1.5.0
HMMER 3.3.2
Horovod 0.21.3
intel-compilers + impi + imkl 2021.2.0
Julia 1.6.1
libfabric 1.12.1
libRmath 4.0.0
LLVM 11.1.0
Mesa 21.1.1
metaWRAP 1.3
Nim 1.4.6
NWChem 7.0.2
OpenBLAS 0.3.15
openCARP 6.0
OpenMM 7.5.0
OpenMPI 4.1.1
Pandoc 2.13
Perl 5.32.1
PMIx 3.2.3
Primer3 2.5.0
PyTorch 1.8.1,
Python 3.9.5
Qt5 5.15.2
Rmath 4.0.4
Rust 1.52.1
SciPy-bundle 2021.05
snakemake 6.1.0
UCX 1.10.0
Valgrind 3.17.0
X11 20210518
YAXT 0.9.0
These changes result from various contributions, made by 19 different
contributors:
* 59 merged pull requests for EasyBuild framework
* 47 merged pull requests for easyblocks
* 349 merged pull requests for easyconfigs
Thanks to everyone who contributed to this release in one way or another!
To upgrade to EasyBuild v4.4.0, there are several options [5].
Two particularly easy options include:
* eb --install-latest-eb-release
* eb --from-pr 13012 # use easyconfig from PR #13012 [6]
Enjoy!
regards,
Kenneth & Miguel
[1] https://pypi.org/project/easybuild/4.4.0
[2]
https://docs.easybuild.io/en/latest/Release_notes.html#easybuild-v4-4-0-june-2nd-2021
[3]
https://docs.easybuild.io/en/latest/Partial_installations.html#installing-additional-extensions-using-k-skip
[4]
https://docs.easybuild.io/en/latest/version-specific/Supported_software.html
[5]
https://docs.easybuild.io/en/latest/Installation.html#updating-an-existing-easybuild-installation
[6] https://github.com/easybuilders/easybuild-easyconfigs/pull/13012
