Re: [easybuild] easybuild new user experience
Hi Malcolm,
On Feb 13, 2015, at 8:01 PM, Malcolm Cook wrote:
> This level of symlink indirection is providing for 'buildsets' whose name
> encodes microprocessor-architecture and creation date of the buildset, only
> one of which is 'current', right?
>
> And, the '-h' and 's' _could_ be spelled out haswell and sandybridge, right?
They _could_ ;-)
> So, if we wanted to also support OSX, cygwin (gasp), we could extend your
> buildset scheme:
>
> ARCHPATH=Linux/sandybridge # or Cygwin/64 or OSX/x86_64 as appropriate
>
> /opt/apps/HPCBIOS/${ARCHPATH}/{MMDDa,MMDDb,etc}
>
> and then for each ARCHPATH, track the 'current'
>
> /opt/apps/HPCBIOS/${ARCHPATH}/current ->
> /opt/apps/HPCBIOS/${ARCHPATH}/YYMM
>
> Right?
Yeap, that’s exactly the idea.
We can argue forever about the exact naming, but the logic is what you
confirmed here.
It gives you the (sometimes needed) freedom to reuse buildsets across archs.
fi. theoretically, you could switch to 32bit modules/builds, from 64bit,
keeping the same namespace (it’s case-specific if that makes any sense;
yet now you _can_ shoot yourself in the foot, with 1 cheap symlink update!)
Best of all, rollback is dirt cheap. Roll-forward for testing? no problem.
N.B. At build time the paths are always explicit (HPCBIOS*DDMM);
it is modulefiles that point to them that make the scheme work.
modulefiles are accessible via symlinks (or farms thereof).
> If so, the part that nags at me, is handling common apps that either are not
> compiled or whose compilation does not depend on this level of detail (i.e.
> x86_64 is enough). Is that what you are suggesting (below) would be handled
> the "symlink farm" (and, for this purpose, can you recommend Gnu stow?).
IMHO, Gnu Stow solves a different problem (when you need to consolidate
multiple /bin, /lib, /usr );
it could be applied to provide for “single-modulefile-bundles” but this is a
totally different use-case,
which only matters when you need to a) reduce length of *PATH* or, b) make
bundled module loads atomic.
In practice, I have never done any scheme more complex than HPCBIOS.DDMM?,
where “?" denotes a node class, because that proved to be sufficient for
clusters
that have passed from my hands. YMMV. Yet, you may implement any scheme ad
infinitum.
I know that our fellows from Vienna had to deal with many platforms and OSes,
at once,
so they may have some insight to offer about their naming practices. However,
my stance is that naming practices will be keep changing as systems change...
... while the tag approach of DDMM? is meant to keep carrying meaning in
the future.
> Thanks so much for you insights. I am new to the administration side of
> HPC. I may be "over-thinking" things. I welcome frank criticism.
Hard fact of life is, we only know if something is “over-designed” after having
applied a technique over multiple years (and sites). Then it’s clear what could
have been thrown out. But in engineering we need provision & safety margins
anyhow,
even if they are never entered to (and sometimes rightly so).
cheers,
Fotis
--
echo "sysadmin know better bash than english" | sed s/min/mins/ \
| sed 's/better bash/bash better/' # signal detected in a CERN forum
Re: [easybuild] easybuild new user experience
Hi,
>
> >> >Then you might create symlinks like:
> >> > /opt/apps/sandybridge -> /opt/apps/*.MMDD
> >> >
> >> >I used a dubious example name above but you get the idea.
>
> Something that has functioned in practice before is a scheme like:
>
> /opt/apps/HPCBIOS.haswell -> /opt/apps/HPCBIOS.MMDD-h
> /opt/apps/HPCBIOS.sandybridge -> /opt/apps/HPCBIOS.MMDD-s
> /opt/apps/HPCBIOS.MMDD-h
> /opt/apps/HPCBIOS.MMDD-s
> /opt/apps/HPCBIOS.MMDD-h
> /opt/apps/HPCBIOS.MMDD-s
> [.. ..]
>
> ie. letters h, s, etc denote node classes which can be treated as a single
> family.
> I think that if you run out of 26 letters, you are a cloud provider, not
> HPC site ;-)
>
> A nice advantage of the above design is that it becomes tractable to
> deliver
> the software trees via packages (fi. rpm), which is not possible in some
> other ways.
>
> Fotis, let's see if I follow you here...
This level of symlink indirection is providing for 'buildsets' whose name
encodes microprocessor-architecture and creation date of the buildset, only
one of which is 'current', right?
And, the '-h' and 's' _could_ be spelled out haswell and sandybridge, right?
So, if we wanted to also support OSX, cygwin (gasp), we could extend your
buildset scheme:
ARCHPATH=Linux/sandybridge # or Cygwin/64 or OSX/x86_64 as appropriate
/opt/apps/HPCBIOS/${ARCHPATH}/{MMDDa,MMDDb,etc}
and then for each ARCHPATH, track the 'current'
/opt/apps/HPCBIOS/${ARCHPATH}/current ->
/opt/apps/HPCBIOS/${ARCHPATH}/YYMM
Right?
If so, the part that nags at me, is handling common apps that either are
not compiled or whose compilation does not depend on this level of detail
(i.e. x86_64 is enough). Is that what you are suggesting (below) would be
handled the "symlink farm" (and, for this purpose, can you recommend Gnu
stow?).
Thanks so much for you insights. I am new to the administration side of
HPC. I may be "over-thinking" things. I welcome frank criticism.
Cheers,
~Malcolm
> On Feb 11, 2015, at 5:09 PM, Cook, Malcolm wrote:
> > Fotis, I note your example name, "sandybridge", apparently encoded an
> intel processor microarchitecture, NOT the name of a linux distribution
> (such as c6 or c7 for releases of centOS, as proposed). I'm trying to
> understand the implications of possibly needing to support a heterogeneous
> environment having multiple CentOS versions (6.5 and 7.x) on multiple core
> types (sandybridge) and would appreciate any more clarity here. Are you
> possibly suggesting that buildsets for each combination of microprocessor
> and OS version might be appropriate
> > (provoking visions of
> /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD )
>
> Yes, technically that is both feasible and cheap (kudos to easybuild for
> that).
> In practice you only need a subset of the combinations, as per your site’s
> reqs.
>
> Then, you can use your traditional OS’s configuration management system
> to have a symlink farm pointing to where each case should have a default,
> as buildsets advance in time and improve. Shoot for both agility and
> stability!
>
> On Feb 11, 2015, at 5:30 PM, Jack Perdue wrote:
> > We certainly have need for such a thing. Our cluster is mostly Ivy
> Bridge
> > (with AVX2 support), including the login nodes where I do most my builds.
> > However, we have some systems (1-2TB) with older chipsets that don't
> > have AVX. So anything built with optarch=True (i.e. -xHost) on the login
> > nodes won't work on the big mem nodes.
>
> Been there done that; the above design should handle this fine.
>
> Sometimes you have the opposite problem: a newish shiny fat node
> basically requires custom optimised builds. Still, solution is the same!
>
> > I could do the build on the big mem (older chipset) nodes but then won't
> > get the benefit of AVX on the newer nodes. So somehow providing a way
> > to distinguish based on the chip set would be kind of nice.
>
> Once upon a time, the famous “target” strings of fi. GCC did the job well:
> x86_64-redhat-linux ## 10 years ago, this was mostly sufficient
> Such names, would be great for symlink farms on buildsets for various
> nodes.
>
> However, on HPC platforms the situation is nowadays more complicated
> because
> architectural families of CPUs require very customised builds w. explicit
> names.
>
> enjoy,
> Fotis
>
> --
> echo "sysadmin know better bash than english" | sed s/min/mins/ \
> | sed 's/better bash/bash better/' # signal detected in a CERN forum
>
>
>
>
>
>
>
>
Re: [easybuild] easybuild new user experience
2015-02-11 17:09 GMT+01:00 Cook, Malcolm :
>
>
> Fotis, I note your example name, "sandybridge", apparently encoded an
> intel processor microarchitecture, NOT the name of a linux distribution
> (such as c6 or c7 for releases of centOS, as proposed). I'm trying to
> understand the implications of possibly needing to support a heterogeneous
> environment having multiple CentOS versions (6.5 and 7.x) on multiple core
> types (sandybridge) and would appreciate any more clarity here. Are you
> possibly suggesting that buildsets for each combination of microprocessor
> and OS version might be appropriate
> (provoking visions of /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD
> )
>
>
>
Hi Malcom,
In case it's helpful, my approach to support different cpu types is to have
a different soft stack by cpu type and then use automounter to access the
right software stack.
As example, I have all my soft stacks in this paths:
/mnt/soft/apps/{sandybridge,nehalem,bulldozer}
Then I use aumounter to "map" /soft/apps to the right folder in
/mnt/soft/apps depending on the cpu type of the machine. In all my compute
nodes the path where the software is accesible is /soft/apps (this is what
my users know about) but automounter is mapping /soft/apps to the right
folder in /mnt/soft/apps/{sandybridge,nehalem,bulldozer}
Of course, for this you need to rebuild all your applications for all the
cpu types you want to support and have a different soft stack for each cpu
but once you have an easyconfig building the application 1 time or 100
times is easy :)
I suppose that adapting that approach to also support multiple
distributions shouldn't be difficult.
regards,
Pablo.
--
Pablo Escobar López
HPC systems engineer
Biozentrum, University of Basel
Swiss Institute of Bioinformatics SIB
Email: pablo.escobarlo...@unibas.ch
Phone: +41 61 267 21 80
http://www.biozentrum.unibas.ch
Re: [easybuild] easybuild new user experience
Hi Malcolm, Jack,
>> >Then you might create symlinks like:
>> > /opt/apps/sandybridge -> /opt/apps/*.MMDD
>> >
>> >I used a dubious example name above but you get the idea.
Something that has functioned in practice before is a scheme like:
/opt/apps/HPCBIOS.haswell -> /opt/apps/HPCBIOS.MMDD-h
/opt/apps/HPCBIOS.sandybridge -> /opt/apps/HPCBIOS.MMDD-s
/opt/apps/HPCBIOS.MMDD-h
/opt/apps/HPCBIOS.MMDD-s
/opt/apps/HPCBIOS.MMDD-h
/opt/apps/HPCBIOS.MMDD-s
[.. ..]
ie. letters h, s, etc denote node classes which can be treated as a single
family.
I think that if you run out of 26 letters, you are a cloud provider, not HPC
site ;-)
A nice advantage of the above design is that it becomes tractable to deliver
the software trees via packages (fi. rpm), which is not possible in some other
ways.
On Feb 11, 2015, at 5:09 PM, Cook, Malcolm wrote:
> Fotis, I note your example name, "sandybridge", apparently encoded an intel
> processor microarchitecture, NOT the name of a linux distribution (such as c6
> or c7 for releases of centOS, as proposed). I'm trying to understand the
> implications of possibly needing to support a heterogeneous environment
> having multiple CentOS versions (6.5 and 7.x) on multiple core types
> (sandybridge) and would appreciate any more clarity here. Are you possibly
> suggesting that buildsets for each combination of microprocessor and OS
> version might be appropriate
> (provoking visions of /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD )
Yes, technically that is both feasible and cheap (kudos to easybuild for that).
In practice you only need a subset of the combinations, as per your site’s reqs.
Then, you can use your traditional OS’s configuration management system
to have a symlink farm pointing to where each case should have a default,
as buildsets advance in time and improve. Shoot for both agility and stability!
On Feb 11, 2015, at 5:30 PM, Jack Perdue wrote:
> We certainly have need for such a thing. Our cluster is mostly Ivy Bridge
> (with AVX2 support), including the login nodes where I do most my builds.
> However, we have some systems (1-2TB) with older chipsets that don't
> have AVX. So anything built with optarch=True (i.e. -xHost) on the login
> nodes won't work on the big mem nodes.
Been there done that; the above design should handle this fine.
Sometimes you have the opposite problem: a newish shiny fat node
basically requires custom optimised builds. Still, solution is the same!
> I could do the build on the big mem (older chipset) nodes but then won't
> get the benefit of AVX on the newer nodes. So somehow providing a way
> to distinguish based on the chip set would be kind of nice.
Once upon a time, the famous “target” strings of fi. GCC did the job well:
x86_64-redhat-linux ## 10 years ago, this was mostly sufficient
Such names, would be great for symlink farms on buildsets for various nodes.
However, on HPC platforms the situation is nowadays more complicated because
architectural families of CPUs require very customised builds w. explicit names.
enjoy,
Fotis
--
echo "sysadmin know better bash than english" | sed s/min/mins/ \
| sed 's/better bash/bash better/' # signal detected in a CERN forum
Re: [easybuild] easybuild new user experience
On Wed, Feb 11, 2015 at 10:30:30AM -0600, Jack Perdue wrote:
> On 02/11/2015 10:09 AM, Cook, Malcolm wrote:
> >Hi Fotis & Stuart,
> >
> >>>What do people do/recommend for multiple OS environments? We are
> > >> currently CentOS 6 but will eventually move to C7. I'm thinking I
> > >> will want a separate application tree for each OS (/projects/app-c6
> > >> and /projects/app-c7).
> > >>
> > >> How do people deal with software with frequent updates (java) or
> > >> security issues? Do you rebuild and remove old packages?
> > >
> > >You may be able to handle both of the above needs,
> > >by using the concept of buildsets, mentioned in p. over here:
> >
> > >https://archive.fosdem.org/2014/schedule/event/hpc_devroom_hpcbios/attachments/slides/491/export/events/attachments/hpc_
> > >devroom_hpcbios/slides/491/FOSDEM14_HPC_devroom_09_HPC_BIOS.pdf
> > >
> > >In principle, the idea is that you create self-contained directory areas
> > >with complete build trees, including modules, at a given point in time.
> > >I've calling them /opt/apps/HPCBIOS.MMDD but any kind of tag will
> > just do.
> > >
> > >Then you might create symlinks like:
> > > /opt/apps/sandybridge -> /opt/apps/*.MMDD
> > >
> > >I used a dubious example name above but you get the idea.
> >
> >Fotis, I note your example name, "sandybridge", apparently encoded an intel
> >processor microarchitecture, NOT the name of a linux distribution (such as
> >c6 or c7 for releases of centOS, as proposed). I'm trying to understand
> >the implications of possibly needing to support a heterogeneous environment
> >having multiple CentOS versions (6.5 and 7.x) on multiple core types
> >(sandybridge) and would appreciate any more clarity here. Are you possibly
> >suggesting that buildsets for each combination of microprocessor and OS
> >version might be appropriate
> >(provoking visions of /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD )
> >
> >??
> We certainly have need for such a thing. Our cluster is mostly Ivy Bridge
> (with AVX2 support), including the login nodes where I do most my builds.
> However, we have some systems (1-2TB) with older chipsets that don't
> have AVX. So anything built with optarch=True (i.e. -xHost) on the login
> nodes won't work on the big mem nodes.
>
> I could do the build on the big mem (older chipset) nodes but then won't
> get the benefit of AVX on the newer nodes. So somehow providing a way
> to distinguish based on the chip set would be kind of nice.
You can do this with modules, could have a look at how Lmod does this.
--
Olav
> ($.02)
>
> jack
>
Re: [easybuild] easybuild new user experience
On 02/11/2015 10:09 AM, Cook, Malcolm wrote:
Hi Fotis & Stuart,
What do people do/recommend for multiple OS environments? We are
>> currently CentOS 6 but will eventually move to C7. I'm thinking I
>> will want a separate application tree for each OS (/projects/app-c6
>> and /projects/app-c7).
>>
>> How do people deal with software with frequent updates (java) or
>> security issues? Do you rebuild and remove old packages?
>
>You may be able to handle both of the above needs,
>by using the concept of buildsets, mentioned in p. over here:
>https://archive.fosdem.org/2014/schedule/event/hpc_devroom_hpcbios/attachments/slides/491/export/events/attachments/hpc_
>devroom_hpcbios/slides/491/FOSDEM14_HPC_devroom_09_HPC_BIOS.pdf
>
>In principle, the idea is that you create self-contained directory areas
>with complete build trees, including modules, at a given point in time.
>I've calling them /opt/apps/HPCBIOS.MMDD but any kind of tag will just
do.
>
>Then you might create symlinks like:
> /opt/apps/sandybridge -> /opt/apps/*.MMDD
>
>I used a dubious example name above but you get the idea.
Fotis, I note your example name, "sandybridge", apparently encoded an intel
processor microarchitecture, NOT the name of a linux distribution (such as c6 or c7 for
releases of centOS, as proposed). I'm trying to understand the implications of
possibly needing to support a heterogeneous environment having multiple CentOS versions
(6.5 and 7.x) on multiple core types (sandybridge) and would appreciate any more clarity
here. Are you possibly suggesting that buildsets for each combination of microprocessor
and OS version might be appropriate
(provoking visions of /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD )
??
We certainly have need for such a thing. Our cluster is mostly Ivy Bridge
(with AVX2 support), including the login nodes where I do most my builds.
However, we have some systems (1-2TB) with older chipsets that don't
have AVX. So anything built with optarch=True (i.e. -xHost) on the login
nodes won't work on the big mem nodes.
I could do the build on the big mem (older chipset) nodes but then won't
get the benefit of AVX on the newer nodes. So somehow providing a way
to distinguish based on the chip set would be kind of nice.
($.02)
jack
RE: [easybuild] easybuild new user experience
Hi Fotis & Stuart,
>> What do people do/recommend for multiple OS environments? We are
>> currently CentOS 6 but will eventually move to C7. I'm thinking I
>> will want a separate application tree for each OS (/projects/app-c6
>> and /projects/app-c7).
>>
>> How do people deal with software with frequent updates (java) or
>> security issues? Do you rebuild and remove old packages?
>
>You may be able to handle both of the above needs,
>by using the concept of buildsets, mentioned in p. over here:
>https://archive.fosdem.org/2014/schedule/event/hpc_devroom_hpcbios/attachments/slides/491/export/events/attachments/hpc_
>devroom_hpcbios/slides/491/FOSDEM14_HPC_devroom_09_HPC_BIOS.pdf
>
>In principle, the idea is that you create self-contained directory areas
>with complete build trees, including modules, at a given point in time.
>I've calling them /opt/apps/HPCBIOS.MMDD but any kind of tag will just do.
>
>Then you might create symlinks like:
> /opt/apps/sandybridge -> /opt/apps/*.MMDD
>
>I used a dubious example name above but you get the idea.
Fotis, I note your example name, "sandybridge", apparently encoded an intel
processor microarchitecture, NOT the name of a linux distribution (such as c6
or c7 for releases of centOS, as proposed). I'm trying to understand the
implications of possibly needing to support a heterogeneous environment having
multiple CentOS versions (6.5 and 7.x) on multiple core types (sandybridge) and
would appreciate any more clarity here. Are you possibly suggesting that
buildsets for each combination of microprocessor and OS version might be
appropriate
(provoking visions of /opt/apps/{sandybridge,Nehalem}/centOS{6,7}/MMDD )
??
~Malcolm
Re: [easybuild] easybuild new user experience
Hi Stuart, On Feb 11, 2015, at 2:32 AM, Stuart Barkley wrote: > I've been looking at easybuild for a little while now and it appears > to be a suitable framework for a good portion of our needs. If you provide software for HPC you are on the right road. > I think I eventually used "--dry-run --robot" and grepped the output > to determine the full dependency lists and create individual 'eb > --stop=fetch --force' commands for every package dependency. I also > used something like this to determine an internal build order so I > didn't need to --robot 50 packages which would all churn and fail on > the same dependency. I think this is the best you can do to pre-download sources nowadays, if case you don’t provide continuous network access, as you described. > What do people do/recommend for multiple OS environments? We are > currently CentOS 6 but will eventually move to C7. I'm thinking I > will want a separate application tree for each OS (/projects/app-c6 > and /projects/app-c7). > > How do people deal with software with frequent updates (java) or > security issues? Do you rebuild and remove old packages? You may be able to handle both of the above needs, by using the concept of buildsets, mentioned in p. over here: https://archive.fosdem.org/2014/schedule/event/hpc_devroom_hpcbios/attachments/slides/491/export/events/attachments/hpc_devroom_hpcbios/slides/491/FOSDEM14_HPC_devroom_09_HPC_BIOS.pdf In principle, the idea is that you create self-contained directory areas with complete build trees, including modules, at a given point in time. I’ve calling them /opt/apps/HPCBIOS.MMDD but any kind of tag will just do. Then you might create symlinks like: /opt/apps/sandybridge -> /opt/apps/*.MMDD I used a dubious example name above but you get the idea. The point is that you modify your symlinks when you transit maintenance windows, so that you don’t break your running jobs. > Do people recommend updating toolchains and the dependencies (e.g. > eliminate multiple versions of python 2.7)? Since this will be our > initial build I don't really need python 2.7.3, 2.7.5 or .2.7.6. > Should I just use the --try options to build for 2.7.8? If you do life sciences/bioinformatics you’ll likely need Python/2.7.3; you can always try to upgrade forward towards whatever, however I’d advise you to do this as a secondary stage, or you’ll be distracted by compatibility issues that raise quickly in these situations. > information about the 1.7 toolchain, I need to review that email > thread. I’m the perpetrator; it’s basically a GCC/4.8.x fresh toolchain, which should serve for a little while. If I was you , I would do the above step and then try something like: eb HPCBIOS_LifeSciences*eb --try-toolchain=goolf,1.7.20 -r That should provide for a quite interesting effect :) Feel free to depart then with other improvements, you may do plenty. > Is there any planned support for beta/test packages? At the present > time I plan to have two separate easybuild installations and after > building/testing an application in the beta installation I'll rebuild > it in the production installation. (With C6 and C7 installations this > might multiply to 4 easybuild installations.) I am afraid that yes, it’s the only way, if you want to play safe. A rationalized way to handle it, is to create “future” buildsets which you test extensively before a maintenance window; once you arrive at the crossing point, you move symlinks around. You can easily roll-back and roll-forward on the cheap, in this way. > Is there any planned support (or something I'm missing) for allowing > someone else to use an existing easybuild installation (including > toolchains and other packages) to build a test/prototype package? I > would like to have other staff be able to build local test packages > off of the production toolchains and be able to give me a set of > proposed/working .eb files for final build. This is similar to the > above beta question, except I don't expect others to be able to > maintain their own easybuild installation (and don't want them writing > into the production installation). Try MODULEPATH=$MODULES_USERAREA:$MODULES_SYSTEMAREA You may also add $MODULES_GROUPAREA in the mix, you get the idea. Many HPC users can actually maintain their easyconfigs, quite well. They are having much bigger problems to solve, most of the time :) If they are python-adverse though, just invite them to write shell scripts and then it’s trivial to convert them to the EB world of things. > Thanks for the work on easybuild. This looks like it will address a > long standing need for building stable versions of software for our > users. The best thanks is sharing easyconfigs ;-) enjoy, Fotis -- echo "sysadmin know better bash than english" | sed s/min/mins/ \ | sed 's/better bash/bash better/' # signal detected in a CERN forum
Re: [easybuild] easybuild new user experience
On Wed, Feb 11, 2015 at 2:32 AM, Stuart Barkley wrote: > I've been looking at easybuild for a little while now and it appears > to be a suitable framework for a good portion of our needs. Welcome to EB! > My cluster does not have direct access to the Internet so I can't use > any of the automated download procedures. I actually consider this a > feature in that I eventually know exactly what software is running and > that nothing was downloaded and installed that wasn't intended. What is stopping your users from copying other software to your cluster and running it? Or using a ssh portforward/socks proxy to get internet on your cluster? > Dependencies/source package downloads: > > I think I eventually used "--dry-run --robot" and grepped the output > to determine the full dependency lists and create individual 'eb > --stop=fetch --force' commands for every package dependency. I also > used something like this to determine an internal build order so I > didn't need to --robot 50 packages which would all churn and fail on > the same dependency. We could create an `--fetch-only` option that doesn't stop on a failed download. > Some questions: > > How do people deal with software with frequent updates (java) or > security issues? Do you rebuild and remove old packages? For the security sensitive package, we use the OS provides ones (openSSL, glibc, ...) and let the OS update them. > Is there any planned support (or something I'm missing) for allowing > someone else to use an existing easybuild installation (including > toolchains and other packages) to build a test/prototype package? I > would like to have other staff be able to build local test packages > off of the production toolchains and be able to give me a set of > proposed/working .eb files for final build. This is similar to the > above beta question, except I don't expect others to be able to > maintain their own easybuild installation (and don't want them writing > into the production installation). You can use a different `-installpath=` for the test packages? If the production toolchains are in the module path, they will be used. > Wish list (maybe for hack-a-thon): > > Don't automatically create $HOME/.local/easybuild (or any other top > level directory). Too many times I forgot to specify my configuration > file name or fumble fingered something and ended up with a mess of > files where I didn't intend. Requiring the top level directory to > already exist prevents a misconfigured easybuild from writing a bunch > of stuff for later cleanup. The error message should be clear about > the directory name so that a simple copy/paste can be used to create > the missing directory when it is truly missing. I don't agree here. Almost all Linux programs will create their own directory under $HOME without asking. It's the proper thing to do. If you have troubles forgetting the specify certain options, add them to the config.cg file? Or export them as bash variables if the change from location to location. > Process for removing a package. It is probably fairly simple, mostly > deleting the installation tree and the modules files. Dependencies > might be tricky (indicating all the dependencies and refuse to remove > would be fine). This is on the wish list for quite some time: https://github.com/hpcugent/easybuild-framework/issues/1000 Ward
Re: [easybuild] easybuild new user experience
Howdy Stuart, I'll offer an unofficial welcome to EasyBuild. I can relate to quite a bit of the below. I'm not sure I have any good answers yet. But WELCOME Your comments will not go unnoticed For example, I just want to second one of your points (while I ponder if I can better answer your other questions/needs than the authors). On 02/10/2015 07:32 PM, Stuart Barkley wrote: Process for removing a package. It is probably fairly simple, mostly deleting the installation tree and the modules files. Dependencies might be tricky (indicating all the dependencies and refuse to remove would be fine). This would be a dream. eb --remove zlib/x.y.z # remove that zlib and everything that depended upon it and/or eb --remove GCC/4.9.0 # remove that toolchain and all modules, and install trees, associated with it I'm curious if Lmod already has a dependency tracking thingie that I can use to script, but if it could be automated into EasyBuild, that would be really sweet (I do a lot of prototyping, especially since working on the Power7). jack
