Hi Cormac,
On 28 Jan 2014, at 22:11, Stijn De Weirdt wrote:
> hi garvey,
>
>
>> There seems to be a restriction in Easybuild that all
>> dependencies/builddependencies defined in a Easybuild
>> configuration file need to be also built with the same toolchain. Is there
>> some way to remove this restriction?
> this issue has been raised before, so we are aware of it, but unless i'm
> mistaken, no easy solution exists for this, and at least i don't consider it
> as a real problem. (but i admit that this is a personal viewpoint, and maybe
> full lmod support might be a good opportunity to introduce a more flexible
> form of subtoolchains).
To add onto Stijn's reply: currently, there's only one way to allow for this,
and you have to modify the easyconfig file(s).
One example:
...
toolchain = {'name': 'goolf', 'version': '1.4.10'}
...
dependencies = [('CMake', '2.8.4', '', ('GCC', '4.7.2'))]
...
The CMake dependency specification indicates that it was built with a toolchain
different from the 'parent', in this case GCC/4.7.2 (which is part of
goolf/1.4.10).
Currently, the EasyBuild framework strictly enforces that dependencies are
either available with the same toolchain as the 'parent', or that another
(compatible!) toolchain was specified explicitly.
We could loosen this up, e.g. by checking for CMake modules built with
goolf/1.4.10, gompi/1.4.10 and GCC/4.7.2 (in that order), following the example
above.
I don't think it's a terrible amount of work, but someone will have to step up
to implement it though.
>
>> e.g
>> If I build ScaLAPACK with the gmvapich2 toolchain and I want to use
>> OpenBLAS(dependencies), I must
>> also build OpenBLAS with the same toolchain gmvapich2. OpenBLAS is not a
>> distributed parallel library and
>> does not need an MPI library, my preference in this case would be to build
>> OpenBLAS with just the GCC
>> compiler and define this version as a dependency when I build ScaLAPACK.
> if you build a toolchain, ScaLAPACK is part of that toolchain. a toolchain
> provides support a c,C++ and fortran compiler; blas, lapack, mpi, scalapack
> and fftw.
> i don't consider it as an issue because it is a form of greatest common
> divisor between software: packages typically need a few of these, and the
> version dependent naming is squased into the toolchain name and version.
> but the important thing: EB supports building with --robot, so it does not
> impose any extra work for the EB user. just do eb --robot
> some_easyconfig_using_some_toolchain.eb and eb will resolve all dependencies
> and build all that is missing (including the whole toolchain).
> the only waste might be that there are duplicate installations, if they are
> truly identical. (eg GCC will only build once for 2 toolchains that are GCC
> based).
>
>>
>> If I build any a distributed parallel Application with CMake and I define
>> CMake as a builddependency,
>> I must build CMAKE with the same parallel toolchain, when I would prefer to
>> build CMAKE with just a GCC
>> compiler.
>>
>> Any suggestions on how to do this?
> no. just set CMAKE as the dependency and build with --robot (and go for a
> coffee ;)
> e.g. some people have tried to convince me that CMAKE is truly independent of
> compiler/blas/mpi/fft, so this could be a waste of compilation time and disk
> usage, but keep in mind that this is the exception (and i'm not convinced
> it's actually correct, given amount of hooks CMAKE provides, also implying an
> inormous amount of patches to be applied, making it very dependent of the
> toolchain; but that's a different discussion altogether.)
>
> mostly just my 2 cents ;)
We have had this discussion before, and it just a difference in point of view,
as Stijn mentioned. I personally do not have a preference for either, I see
benefits in both.
This is more or less a 'religious' discussion, that can probably be avoided in
the future if the framework checks for dependencies with multiple toolchains,
hierarchically.
We have some documentation on this issue as well, see
https://github.com/hpcugent/easybuild/wiki/Minimal-vs-full-toolchains .
regards,
Kenneth
