Hi all,many thanks for your insights, this is all really interesting and helps me better explain this to our users :)
Concerning Qt, I see that PyQt relies on the full toolchain because of Python, but all the Qt5 easyconfigs in the official repo are also using full toolchains:
https://github.com/easybuilders/easybuild-easyconfigs/tree/master/easybuild/easyconfigs/q/Qt5 even though Python is not a dependency for these.Couldn't at least this be changed to GCCcore then, or would this lead to trouble for people not using --minimal-toolchains?
Best, Maik Am 21.06.2018 um 01:54 schrieb Bart Oldeman:
Hi Maik,there is already an in-between solution for Python that sits at the compiler level using the "golf" toolchain which is a subset of "foss" (everything except MPI, so GCC + OpenBLAS + LAPACK + FFTW, but no ScaLAPACK and no FFTW with MPI).https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/p/Python/Python-3.6.4-golf-2018a.eb it's identical to foss-2018a except for mpi4py. I've got some PRs in transit to get the remaining pieces in there: iimkl + Python 3.6.4 built with it (subset of "intel") https://github.com/easybuilders/easybuild-easyconfigs/pull/6092 which depends on this easyblock PR: https://github.com/easybuilders/easybuild-easyblocks/pull/1393 (which Kenneth reviewed but then fell through the cracks) and last but not least my toolchain PRs to allow multiple subtoolchains: https://github.com/easybuilders/easybuild-framework/pull/2466 this allows a lot of sharing e.g. for goolfc (=fosscuda): goolf => subtoolchains: gompi and golf goolfc => subtoolchains: gompic and golfc golfc => subtoolchains: gcccuda and golfso with this structure anything compiled without MPI and without CUDA, but with linear algebra, can be shared.Cheers, BartOn Wed, 20 Jun 2018 at 08:59, Vanzo, Davide <[email protected] <mailto:[email protected]>> wrote:Maik, I double Jack's explanation on why QT5 is built at the highest toolchain level. Similarly in our stack we moved Python to the GCC/iccifort level by removing the packages that depend on MPI/FFTW and install Qt5 with the GCC/iccofort toolchains. Moving Python to a lower toolchain in the official easyconfig repo is something that we have been discussing for a while and we are making steps in that directions. In the meantime nothing stops you to do are we and Jack did. As for CUDA, the use of toolchains like `fosscuda` and `intelcuda` allow to avoid the need of adding version suffixes for MPI and all other software built with CUDA support. In our case we prefer to avoid version suffixes as much as we can since they introduce unnecessary complication for our users. Obviously this is possible only if you are using hierarchical modules.-- *Davide Vanzo, PhD*/Application Developer/ /Adjunct Assistant Professor of Chemical and Biomolecular Engineering/ Advanced Computing Center for Research and Education (ACCRE) Vanderbilt University - Hill Center 201 (615)-875-9137 www.vanderbilt.edu/accre <http://www.vanderbilt.edu/accre> On 2018-06-20 10:19:58-05:00 [email protected] <mailto:[email protected]> wrote: [oops.... meant to send to the list] -------- Forwarded Message -------- Subject: Re: [easybuild] Minimal vs full toolchain, Qt, CUDA etc. Date: Wed, 20 Jun 2018 09:56:06 -0500 From: Jack Perdue &lt;[email protected] <mailto:amp%3blt%[email protected]>&gt; To: Maik Schmidt &lt;[email protected] <mailto:amp%3blt%[email protected]>&gt; Howdy Maik, Here we use a stripped down Python as a builddependency to build: &amp;nbsp; Qt5-5.10.1-GCCcore-6.4.0-Python-2.7.14-bare.eb and &amp;nbsp; Qt5-5.10.1-GCCcore-6.4.0-Python-3.6.3-bare.eb By default, the full Python needs MPI/maths (for numpy) so if you build Qt with the regular Python you have to promote the toolchain.&amp;nbsp; Python-bare just provides the basics. Works fairly well. There are other initiatives in EB to clean this up using Python-core and the like.&amp;nbsp; But this is what we've been using for now. As for CUDA.... I wondered the same.... the answer was that OpenMPI has hooks for CUDA so if you include CUDA early in the toolchain (while building OpenMPI) then you get some MPI-enabled CUDA.&amp;nbsp; I/we haven't much experience with that yet (curious to see what TensorFlow can do) but that's the reasoning for that (though I do wish it was easier like you suggest) As ever, there are examples at: &amp;nbsp;https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.siliconslick.com%2Feasybuild%2Feasyconfigs%2F&amp;amp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C969e7c72f65249da661e08d5d6c145e8%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636651047945281105&amp;amp;sdata=uZgbznCATjG8G%2BC6o5Pxkvz1UtS2qIJ8NiLSqYVn9Ys%3D&amp;amp;reserved=0 See ada and terra. Jack Perdue Lead Systems Administrator High Performance Research Computing TAMU Division of Research [email protected] <mailto:[email protected]> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fhprc.tamu.edu&amp;amp;data=02%7C01%7Cdavide.vanzo%40vanderbilt.edu%7C969e7c72f65249da661e08d5d6c145e8%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C1%7C636651047945281105&amp;amp;sdata=7DXnIKRWuF6BXcxiI1YSxFxdv7M8W8Gdb116krNrxUQ%3D&amp;amp;reserved=0 HPRC Helpdesk:[email protected] <mailto:[email protected]> On 06/20/2018 09:21 AM, Maik Schmidt wrote: &amp;gt; Hi, I've been asked by one of our users why Qt5 is built with the full &amp;gt; toolchain (foss or intel) even though it does not really use MPI or &amp;gt; MKL for that matter. I've looked at the dependencies of Qt itself and &amp;gt; apparently he is correct, why is this not GCCcore? There's no reason &amp;gt; to use the full toolchain here, right? &amp;gt; &amp;gt; On another note, what has been the reasoning behind introducing an &amp;gt; entire new toolchain only to add CUDA? it really makes not much sense &amp;gt; to me, because then I have to build a lot of duplicate software that &amp;gt; doesn't even need CUDA just to support this toolchain (foss vs &amp;gt; fosscuda)... e.g. why would I need a HDF5 -fosscuda if it is exactly &amp;gt; the same as the -foss version? &amp;gt; &amp;gt; The solution with just adding a versionsuffix and CUDA as a dependency &amp;gt; to software requiring it seemed much cleaner to me, but maybe I'm &amp;gt; missing something here... &amp;gt; &amp;gt; Thanks for your input, &amp;gt; &amp;gt; Maik &amp;gt; &lt;/[email protected] <mailto:[email protected]>&gt;&lt;/[email protected] <mailto:[email protected]>&gt; --Dr. Bart E. Oldeman | [email protected] <mailto:[email protected]> | [email protected] <mailto:[email protected]>Scientific Computing Analyst / Analyste en calcul scientifiqueMcGill HPC Centre / Centre de Calcul Haute Performance de McGill | http://www.hpc.mcgill.caCalcul Québec | http://www.calculquebec.ca Compute/Calcul Canada | http://www.computecanada.ca Tel/Tél: 514-396-8926 | Fax/Télécopieur: 514-396-8934
-- Maik Schmidt HPC Services Technische Universität Dresden Zentrum für Informationsdienste und Hochleistungsrechnen (ZIH) Willers-Bau A116 D-01062 Dresden Telefon: +49 351 463-32836
smime.p7s
Description: S/MIME Cryptographic Signature
