Installing R extensions with the ext_lists option seems to work well for us
too. However, at the moment it seems not possible to
pass any config parameters to ext package to be built. It would be good if the
self.configurevars is defined as an extra parameter
for the eblock RPackage. Or is there any other way, except patches, to have an
access to ext package configuration ?
.....
class RPackage(ExtensionEasyBlock):
..
self.configurevars = []
self.configureargs = []
Best wishes,
Gizo
>
>
> On 17/12/2016 04:54, Jack Perdue wrote:
> > On 12/16/2016 04:37 PM, Jens Timmerman wrote:
> >> Hi,
> >>
> >>
> >> On 16/12/2016 20:44, Siddiqui, Shahzeb wrote:
> >>> It comes down to whether we want simplicity in R, Python, or Perl
> >>> installation by having packages installed inside the easyconfig
> >>> file. It works well for most packages assuming there is no issue
> >>> during build. There are cases when some package don't build properly
> >>> and having to troubleshoot them with an entire list of packages
> >>> become difficult.
> >>>
> >>> I would like to decouple the R packages into individual easyconfig
> >>> file, because I can resolve any dependencies and work on installing
> >>> the particular package. Even if the process might be repetitive for
> >>> most builds, at least the R collection will be built correctly.
> >>> Another benefit is that I would like to package into RPMs, and it
> >>> would be beneficial to create a RPM for each R package rather than
> >>> just the main R. Just like how RHEL provides Python RPMs for its
> >>> base and Python related packages, I think we can achieve this with
> >>> Easybuild.
> >>>
> >>> I also need packages from Bioconductor and would like this in
> >>> Easybuild.
> >>>
> >>>
> >> You can already do this, just use the RPackage easyblock:
> >> https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/rpackage.py
> >> see for example all these easyconfigs:
> >> https://github.com/hpcugent/easybuild-easyconfigs/search?utf8=%E2%9C%93&q=rpackage
> >>
> >>
> >> As you will see these already use packages from Bioconductor etc.
> >>
> >> In fact this was the first way to install R packages in EasyBuild,
> >> the ext_lists option was only added later.
> >> This Rpackage EasyBlock will install the R package like a user would
> >> normally do, and generate a module with an extra R_LIBS environment
> >> variable so R will be able to find it.
> >>
> >>
> >> Regards,
> >> Jens Timmerman
> >
> > Yeah... but there's a lot to be said for a) breaking it down to
> > individual blocks with known version numbers (for tracking) so you can
> > debug
> > them individually
> Yes, that's exactly what is happening here. See the lists of easyconfigs
> I link to, a lot of seperate ones for just one R package
> > and b) avoiding having to wait an hour on a
> > 28-core system for your EB-driven R build to fail because some extension
> > had a problem.
> >
>
>
> As for having your R build fail beacause 1 extension has a problem, I
> always worked around this by first doing a `bare` R install
> which would generate a working R module, and then add the ext_list to
> the eb and run it again with `--keep`
>
> This would keep the current R build and install the R extensions, if one
> failed to install the previously installed ones would still be there.
>
>
> Regards,
> Jens
> > For our R person here (who supports the R users), I install a fairly bare
> > version that allows for PIC offloading (via MKL) and he does all the
> > extensions.
> >
> > As for PythonPackage installing .eggs from other packages without
> > warning me, I'm going to have to talk to somebody. I don't like it.
> >
> > jack
> >
> >>> -----Original Message-----
> >>> From: easybuild-requ...@lists.ugent.be
> >>> [mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Jack Perdue
> >>> Sent: Friday, December 16, 2016 1:38 PM
> >>> To: easybuild@lists.ugent.be
> >>> Subject: Re: [easybuild] R extra packages
> >>>
> >>> On 12/16/2016 10:54 AM, Siddiqui, Shahzeb wrote:
> >>>> Hi Jack,
> >>>>
> >>>> I checked out your work, this is great work. Did you consider
> >>>> sending your easyconfig files to EB. I think your collection of
> >>>> easyconfig files is a good starting point for building out the R
> >>>> collection. I think it is imperative to have R collection by
> >>>> version because these R packages change along with the version of R.
> >>>>
> >>>> I would like if we can make a R collection for R 3.3.1 which is the
> >>>> latest one.
> >>> If anything, I would send the script(s) to EB... I wouldn't want to
> >>> swamp them with any more easyconfigs than I haven't already swamped
> >>> them with (got lots that I need to send in).
> >>>
> >>> But I would want to do Python and Perl too. And I'd want support
> >>> for things like Bioconducter and whoever.
> >>>
> >>> It wasn't real popular here but we tend to make things easy for
> >>> users (to not confuse them).
> >>> Making it easy for ourselves seems to be another argument (hard to
> >>> disagree with the ease of "pip install").
> >>>
> >>> But if I was doing performance analysis like I've done in the past
> >>> I'd want to nail down every version I'm using and make sure that
> >>> some random update didn't warp my data.
> >>>
> >>> jack
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: easybuild-requ...@lists.ugent.be
> >>>> [mailto:easybuild-requ...@lists.ugent.be] On Behalf Of Jack Perdue
> >>>> Sent: Friday, December 16, 2016 11:39 AM
> >>>> To: easybuild@lists.ugent.be
> >>>> Subject: Re: [easybuild] R extra packages
> >>>>
> >>>> On 12/16/2016 10:22 AM, Siddiqui, Shahzeb wrote:
> >>>>> Hello,
> >>>>>
> >>>>> I would like to create a easyconfig for each R package, I find that
> >>>>> to be best suitable for our environment at Pfizer. I know we can pass
> >>>>> along the R packages in the R easybuild using exts_list . The only
> >>>>> issue is we have our own set of R packages that we need to install,
> >>>>> and I am not sure in which order they need to be installed.
> >>>>>
> >>>>> It's important to separate the R installation from the R packages
> >>>>> because in the case of a build error in R package it would fail to
> >>>>> build R if they are both together.
> >>>>>
> >>>>> I would like to build R packages as modules which will be
> >>>>> particularly important for some special R packages that require a
> >>>>> special dependency. For instance we need R2Jags and that has a
> >>>>> dependency for JAGS. I have JAGS in a module environment so I would
> >>>>> like R2Jags module file to load Jags before a user can use R2Jags.
> >>>>>
> >>>>> When loading base R module it will allow user to have access to base
> >>>>> R packages that is quite small. The special R packages that are build
> >>>>> with Easybuild will have their R module only visible when loading R.
> >>>>> This can be done via MODULEPATH variable.
> >>>>>
> >>>>> Similar to how you have Python package like numpy, I would like this
> >>>>> framework for R packages.
> >>>>>
> >>>> If you want to look over my experiment for automatically building
> >>>> easyconfigs based on CRAN data, see:
> >>>>
> >>>> http://www.siliconslick.com/easybuild/R_madness/
> >>>>
> >>>> There are the scripts (which will need work), the easyconfigs
> >>>> created, and the modules built from those.
> >>>>
> >>>> I thought it was a great idea. Others on our staff, not so much
> >>>> (they are content to use R and Python's built in tools for
> >>>> installing to a directory out of the EasyBuild tree).
> >>>>
> >>>> Personally, as someone who likes to keep track of what versions he's
> >>>> using (and not have some "pip" update something under my feet), I'd
> >>>> spend time doing the same for Python... but I got plenty other stuff
> >>>> to keep me busy [I'm having a good old time with --minimal-toolchains
> >>>> and my new 2016D toolchains based on GCCcore/6.2.0]
> >>>>
> >>>> jack
> >>>>
> >>>>> Shahzeb Siddiqui
> >>>>>
> >>>>> HPC Linux Engineer
> >>>>>
> >>>>> B2220-447.2
> >>>>>
> >>>>> Groton, CT
> >>>>>
> >>>>>
> >>
> >
>
