Hi,
On 16/12/2016 20:44, Siddiqui, Shahzeb wrote:
It comes down to whether we want simplicity in R, Python, or Perl installation
by having packages installed inside the easyconfig file. It works well for most
packages assuming there is no issue during build. There are cases when some
package don't build properly and having to troubleshoot them with an entire
list of packages become difficult.
I would like to decouple the R packages into individual easyconfig file,
because I can resolve any dependencies and work on installing the particular
package. Even if the process might be repetitive for most builds, at least the
R collection will be built correctly. Another benefit is that I would like to
package into RPMs, and it would be beneficial to create a RPM for each R
package rather than just the main R. Just like how RHEL provides Python RPMs
for its base and Python related packages, I think we can achieve this with
Easybuild.
I also need packages from Bioconductor and would like this in Easybuild.
You can already do this, just use the RPackage easyblock:
https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/rpackage.py
see for example all these easyconfigs:
https://github.com/hpcugent/easybuild-easyconfigs/search?utf8=%E2%9C%93&q=rpackage
As you will see these already use packages from Bioconductor etc.
In fact this was the first way to install R packages in EasyBuild, the
ext_lists option was only added later.
This Rpackage EasyBlock will install the R package like a user would
normally do, and generate a module with an extra R_LIBS environment
variable so R will be able to find it.
Regards,
Jens Timmerman
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Jack Perdue
Sent: Friday, December 16, 2016 1:38 PM
To: [email protected]
Subject: Re: [easybuild] R extra packages
On 12/16/2016 10:54 AM, Siddiqui, Shahzeb wrote:
Hi Jack,
I checked out your work, this is great work. Did you consider sending your
easyconfig files to EB. I think your collection of easyconfig files is a good
starting point for building out the R collection. I think it is imperative to
have R collection by version because these R packages change along with the
version of R.
I would like if we can make a R collection for R 3.3.1 which is the latest one.
If anything, I would send the script(s) to EB... I wouldn't want to swamp them
with any more easyconfigs than I haven't already swamped them with (got lots
that I need to send in).
But I would want to do Python and Perl too. And I'd want support for things
like Bioconducter and whoever.
It wasn't real popular here but we tend to make things easy for users (to not
confuse them).
Making it easy for ourselves seems to be another argument (hard to disagree with the ease
of "pip install").
But if I was doing performance analysis like I've done in the past I'd want to
nail down every version I'm using and make sure that some random update didn't
warp my data.
jack
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Jack Perdue
Sent: Friday, December 16, 2016 11:39 AM
To: [email protected]
Subject: Re: [easybuild] R extra packages
On 12/16/2016 10:22 AM, Siddiqui, Shahzeb wrote:
Hello,
I would like to create a easyconfig for each R package, I find that
to be best suitable for our environment at Pfizer. I know we can pass
along the R packages in the R easybuild using exts_list . The only
issue is we have our own set of R packages that we need to install,
and I am not sure in which order they need to be installed.
It's important to separate the R installation from the R packages
because in the case of a build error in R package it would fail to
build R if they are both together.
I would like to build R packages as modules which will be
particularly important for some special R packages that require a
special dependency. For instance we need R2Jags and that has a
dependency for JAGS. I have JAGS in a module environment so I would
like R2Jags module file to load Jags before a user can use R2Jags.
When loading base R module it will allow user to have access to base
R packages that is quite small. The special R packages that are build
with Easybuild will have their R module only visible when loading R.
This can be done via MODULEPATH variable.
Similar to how you have Python package like numpy, I would like this
framework for R packages.
If you want to look over my experiment for automatically building easyconfigs
based on CRAN data, see:
http://www.siliconslick.com/easybuild/R_madness/
There are the scripts (which will need work), the easyconfigs created, and the
modules built from those.
I thought it was a great idea. Others on our staff, not so much (they are
content to use R and Python's built in tools for installing to a directory out
of the EasyBuild tree).
Personally, as someone who likes to keep track of what versions he's
using (and not have some "pip" update something under my feet), I'd
spend time doing the same for Python... but I got plenty other stuff
to keep me busy [I'm having a good old time with --minimal-toolchains
and my new 2016D toolchains based on GCCcore/6.2.0]
jack
Shahzeb Siddiqui
HPC Linux Engineer
B2220-447.2
Groton, CT
