Dear Jakob,
On 08/04/2019 11:48, Jakob Schiøtz wrote:
Hi,
What is the status for Python in the 2019a toolchains? I see no Python
easyconfig for the foss/2019a and intel/2019a toolchains, but there is one for
GCCcore-8.2.0, which I assume is the common root toolchain for the two others.
That one does not contain the usual numpy and scipy modules, presumably because
they benefit from being in the full toolchains. But where do they live now, in
some scientific python bundle, I presume?
We've indeed made a couple of changes w.r.t. how Python is handled to
the 2019a generation of easyconfigs (and onwards).
I'm planning to thoroughly document this soon after the next EasyBuild
release (which should happen this week).
In short: the Python installation as it was done before (with foss/intel
toolchain and including extensions like numpy/scipy) is now split up in
two easyconfigs:
(1) A Python easyconfig (one for Python 2.x, one for Python 3.x)
using GCCcore as toolchain, incl. several Python packages as extensions
(common things like pip, nose, requests, pytest, future).
This change was made to "push down" Python in the hierarchy, so we
can reuse the same Python easyconfig for several toolchains (foss,
intel, iomkl, etc.).
See for example
https://github.com/easybuilders/easybuild-easyconfigs/blob/develop/easybuild/easyconfigs/p/Python/Python-3.7.2-GCCcore-8.2.0.eb
.
We decided to go forward with Python 3.7.2 (rather than sticking
with Python 3.6.x) since the problems we had before with Python 3.7 seem
to be largely resolved.
(2) A SciPy-bundle easyconfig that installs a bunch of additional
extensions like numpy/scipy/pandas/mpi4py on top of Python, which
require a "full" toolchain (incl. MPI, BLAS, LAPACK).
See
https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/s/SciPy-bundle
.
One important detail here is that we only have two SciPy-bundle
easyconfigs: one using the foss toolchain, one using intel.
Each of them installs these additional extensions for *both* Python
2.x and 3.x, in a single installation prefix (using the new "multi_deps"
support that was added to the EasyBuild framework).
By default Python 3.x is loaded together with it, but you can
either load Python 2.x first, or swap to it after loading the
SciPy-bundle module if you prefer or need to use Python 2.x
A more elaborate motivation & overview will be added to the EasyBuild
documentation soon (I hope).
Do let us know if you have any questions on this!
regards,
Kenneth
Best regards
Jakob
--
Jakob Schiøtz, professor, Ph.D.
Department of Physics
Technical University of Denmark
DK-2800 Kongens Lyngby, Denmark
http://www.fysik.dtu.dk/~schiotz/
