Personally I would not recommend the approach of adding a dedicated easyconfig for each R package. I think it would be hard to maintain specially when defining the dependency chain.
Have you look at this? https://github.com/fgeorgatos/easybuild.experimental/tree/master/users/pneerincx Using that script you can install R to you home folder using easybuild, then load R from your home, install all the required R packages using the standard R tools which will take care of dependencies resolution and then generate a R easyconfig which would include all the R packages in the " exts_list" regards, Pablo. 2016-12-17 4:54 GMT+01:00 Jack Perdue <j-per...@tamu.edu>: > On 12/16/2016 04:37 PM, Jens Timmerman wrote: > > Hi, > > > > > > On 16/12/2016 20:44, Siddiqui, Shahzeb wrote: > >> It comes down to whether we want simplicity in R, Python, or Perl > >> installation by having packages installed inside the easyconfig file. > >> It works well for most packages assuming there is no issue during > >> build. There are cases when some package don't build properly and > >> having to troubleshoot them with an entire list of packages become > >> difficult. > >> > >> I would like to decouple the R packages into individual easyconfig > >> file, because I can resolve any dependencies and work on installing > >> the particular package. Even if the process might be repetitive for > >> most builds, at least the R collection will be built correctly. > >> Another benefit is that I would like to package into RPMs, and it > >> would be beneficial to create a RPM for each R package rather than > >> just the main R. Just like how RHEL provides Python RPMs for its base > >> and Python related packages, I think we can achieve this with Easybuild. > >> > >> I also need packages from Bioconductor and would like this in Easybuild. > >> > >> > > You can already do this, just use the RPackage easyblock: > > https://github.com/hpcugent/easybuild-easyblocks/blob/ > master/easybuild/easyblocks/generic/rpackage.py > > see for example all these easyconfigs: > > https://github.com/hpcugent/easybuild-easyconfigs/search? > utf8=%E2%9C%93&q=rpackage > > > > > > As you will see these already use packages from Bioconductor etc. > > > > In fact this was the first way to install R packages in EasyBuild, the > > ext_lists option was only added later. > > This Rpackage EasyBlock will install the R package like a user would > > normally do, and generate a module with an extra R_LIBS environment > > variable so R will be able to find it. > > > > > > Regards, > > Jens Timmerman > > Yeah... but there's a lot to be said for a) breaking it down to > individual blocks with known version numbers (for tracking) so you can > debug > them individually and b) avoiding having to wait an hour on a > 28-core system for your EB-driven R build to fail because some extension > had a problem. > > For our R person here (who supports the R users), I install a fairly bare > version that allows for PIC offloading (via MKL) and he does all the > extensions. > > As for PythonPackage installing .eggs from other packages without > warning me, I'm going to have to talk to somebody. I don't like it. > > jack > > >> -----Original Message----- > >> From: easybuild-requ...@lists.ugent.be > >> [mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Jack Perdue > >> Sent: Friday, December 16, 2016 1:38 PM > >> To: easybuild@lists.ugent.be > >> Subject: Re: [easybuild] R extra packages > >> > >> On 12/16/2016 10:54 AM, Siddiqui, Shahzeb wrote: > >>> Hi Jack, > >>> > >>> I checked out your work, this is great work. Did you consider > >>> sending your easyconfig files to EB. I think your collection of > >>> easyconfig files is a good starting point for building out the R > >>> collection. I think it is imperative to have R collection by version > >>> because these R packages change along with the version of R. > >>> > >>> I would like if we can make a R collection for R 3.3.1 which is the > >>> latest one. > >> If anything, I would send the script(s) to EB... I wouldn't want to > >> swamp them with any more easyconfigs than I haven't already swamped > >> them with (got lots that I need to send in). > >> > >> But I would want to do Python and Perl too. And I'd want support for > >> things like Bioconducter and whoever. > >> > >> It wasn't real popular here but we tend to make things easy for users > >> (to not confuse them). > >> Making it easy for ourselves seems to be another argument (hard to > >> disagree with the ease of "pip install"). > >> > >> But if I was doing performance analysis like I've done in the past > >> I'd want to nail down every version I'm using and make sure that some > >> random update didn't warp my data. > >> > >> jack > >> > >> > >>> -----Original Message----- > >>> From: easybuild-requ...@lists.ugent.be > >>> [mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Jack Perdue > >>> Sent: Friday, December 16, 2016 11:39 AM > >>> To: easybuild@lists.ugent.be > >>> Subject: Re: [easybuild] R extra packages > >>> > >>> On 12/16/2016 10:22 AM, Siddiqui, Shahzeb wrote: > >>>> Hello, > >>>> > >>>> I would like to create a easyconfig for each R package, I find that > >>>> to be best suitable for our environment at Pfizer. I know we can pass > >>>> along the R packages in the R easybuild using exts_list . The only > >>>> issue is we have our own set of R packages that we need to install, > >>>> and I am not sure in which order they need to be installed. > >>>> > >>>> It's important to separate the R installation from the R packages > >>>> because in the case of a build error in R package it would fail to > >>>> build R if they are both together. > >>>> > >>>> I would like to build R packages as modules which will be > >>>> particularly important for some special R packages that require a > >>>> special dependency. For instance we need R2Jags and that has a > >>>> dependency for JAGS. I have JAGS in a module environment so I would > >>>> like R2Jags module file to load Jags before a user can use R2Jags. > >>>> > >>>> When loading base R module it will allow user to have access to base > >>>> R packages that is quite small. The special R packages that are build > >>>> with Easybuild will have their R module only visible when loading R. > >>>> This can be done via MODULEPATH variable. > >>>> > >>>> Similar to how you have Python package like numpy, I would like this > >>>> framework for R packages. > >>>> > >>> If you want to look over my experiment for automatically building > >>> easyconfigs based on CRAN data, see: > >>> > >>> http://www.siliconslick.com/easybuild/R_madness/ > >>> > >>> There are the scripts (which will need work), the easyconfigs > >>> created, and the modules built from those. > >>> > >>> I thought it was a great idea. Others on our staff, not so much > >>> (they are content to use R and Python's built in tools for > >>> installing to a directory out of the EasyBuild tree). > >>> > >>> Personally, as someone who likes to keep track of what versions he's > >>> using (and not have some "pip" update something under my feet), I'd > >>> spend time doing the same for Python... but I got plenty other stuff > >>> to keep me busy [I'm having a good old time with --minimal-toolchains > >>> and my new 2016D toolchains based on GCCcore/6.2.0] > >>> > >>> jack > >>> > >>>> Shahzeb Siddiqui > >>>> > >>>> HPC Linux Engineer > >>>> > >>>> B2220-447.2 > >>>> > >>>> Groton, CT > >>>> > >>>> > > > > -- Pablo Escobar López HPC systems engineer sciCORE, University of Basel SIB Swiss Institute of Bioinformatics http://scicore.unibas.ch
