Dear Kenneth, Thank you very much! It looks like a sensible way to handle it.
I think I will have to wait till the new EasyBuild framework is released and installed here, before I can proceed with this. Then I will make builds of our software for 2019a. Best regards Jakob > On 8 Apr 2019, at 16:43, Kenneth Hoste <kenneth.ho...@ugent.be> wrote: > > Dear Jakob, > > On 08/04/2019 11:48, Jakob Schiøtz wrote: >> Hi, >> What is the status for Python in the 2019a toolchains? I see no Python >> easyconfig for the foss/2019a and intel/2019a toolchains, but there is one >> for GCCcore-8.2.0, which I assume is the common root toolchain for the two >> others. That one does not contain the usual numpy and scipy modules, >> presumably because they benefit from being in the full toolchains. But >> where do they live now, in some scientific python bundle, I presume? > > > We've indeed made a couple of changes w.r.t. how Python is handled to the > 2019a generation of easyconfigs (and onwards). > I'm planning to thoroughly document this soon after the next EasyBuild > release (which should happen this week). > > In short: the Python installation as it was done before (with foss/intel > toolchain and including extensions like numpy/scipy) is now split up in two > easyconfigs: > > (1) A Python easyconfig (one for Python 2.x, one for Python 3.x) using > GCCcore as toolchain, incl. several Python packages as extensions (common > things like pip, nose, requests, pytest, future). > > This change was made to "push down" Python in the hierarchy, so we can > reuse the same Python easyconfig for several toolchains (foss, intel, iomkl, > etc.). > > See for example > https://github.com/easybuilders/easybuild-easyconfigs/blob/develop/easybuild/easyconfigs/p/Python/Python-3.7.2-GCCcore-8.2.0.eb > . > We decided to go forward with Python 3.7.2 (rather than sticking with > Python 3.6.x) since the problems we had before with Python 3.7 seem to be > largely resolved. > > > (2) A SciPy-bundle easyconfig that installs a bunch of additional extensions > like numpy/scipy/pandas/mpi4py on top of Python, which require a "full" > toolchain (incl. MPI, BLAS, LAPACK). > > See > https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/s/SciPy-bundle > . > > One important detail here is that we only have two SciPy-bundle > easyconfigs: one using the foss toolchain, one using intel. > Each of them installs these additional extensions for *both* Python 2.x > and 3.x, in a single installation prefix (using the new "multi_deps" support > that was added to the EasyBuild framework). > By default Python 3.x is loaded together with it, but you can either load > Python 2.x first, or swap to it after loading the SciPy-bundle module if you > prefer or need to use Python 2.x > > A more elaborate motivation & overview will be added to the EasyBuild > documentation soon (I hope). > > Do let us know if you have any questions on this! > > > regards, > > Kenneth >> Best regards >> Jakob >> -- >> Jakob Schiøtz, professor, Ph.D. >> Department of Physics >> Technical University of Denmark >> DK-2800 Kongens Lyngby, Denmark >> http://www.fysik.dtu.dk/~schiotz/ -- Jakob Schiøtz, professor, Ph.D. Department of Physics Technical University of Denmark DK-2800 Kongens Lyngby, Denmark http://www.fysik.dtu.dk/~schiotz/
