Re: [Rd] shared libraries: missing soname
Dirk,
Dirk Eddelbuettel writes:
> You assume that change == breakage. Yet that assumption is baseless.
> Which is what someone like Martin (R Core, and "at it" since the 80s pre-R
> and 90s with the almost very beginning of R) and myself (around R since the
> late 90s, somewhat involved since the early 00s) keep telling you.
> At some point it might appear to be approproiiate for you to actually take
> our word for it.
It is not that I am not taking your word. There are some unique (to me)
approaches in R and some of them are subtle. I am also taking criticism from
the other side, because, as I said, some conventions collide. When Martin said,
> All these libraries "belong to R" and are tied to a specific version of R...
I understood that "tied to a specific version of R" meant that (Debian/FreeBSD)
R packages should be updated in lock step with R. So, on Debian, changes in the
r-core package never necessitate a bump of r-cran-*? In other words, the
libR.so interface is guaranteed to be stable across releases?
> | I also notice that on Debian you make a soft link of /usr/lib/R/lib/libR.so
> to
> | /usr/lib/libR.so. Given all that has been discussed, I am unclear why.
> Well noticed -- yet a stricly personaly reason via two projects I have been
> (co-)authoring: littler and RInside. They both "embed" R via libR and we do
> both rpath ("somewhat" verboten by Debian Policy as it hard codes a path,
> hence the alternate of placing it where ldd / ldconfig find it).
> But please note that that is _me_ doing this, and the R Core gospel we have
> been trying for you to understand still stand: __what you insist is needed
> actually is not__.
Understood.
> | Thank you for sticking with the thread,
> Sorry for coming through as pedantic but you (and we're now at what, six
> posts in and counting?) still start from the wrong (at least outside of the
> gilded confines of FreeBSD) premise. It. Just. Works.
> Just give us and the unknown-but-sometimes-estimated-to-be-in-the-millions of
> R users some credit. What is there __works__. Seriously. Debian folks are
> pendantic for technical excellence __and even they have no issue with
> per-package and local shared libraries__. Which is what this is. Really.
Pedantic is fine. I give credit! I would not be investing my (comparatively
minuscule) time if I did not appreciate R. I would also not be doing anyone a
service if I released a flawed FreeBSD package that did not do the project
justice.
Joseph
signature.asc
Description: PGP signature
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
On 24 November 2016 at 22:09, Joseph Mingrone wrote:
| For a frequently-updated Debian package repository, I assume you _manually_
bump
| all the Debian R packages whenever the main R package is upgraded. Is that
| correct? That is, when a Debian user upgrades the r-base package from, say,
| version 3.2.5 to 3.3.1, then r-cran-tseries must/should be rebuilt/reinstalled
| against the new R package? If so, maybe there is something useful we could
You assume that change == breakage. Yet that assumption is baseless.
Which is what someone like Martin (R Core, and "at it" since the 80s pre-R
and 90s with the almost very beginning of R) and myself (around R since the
late 90s, somewhat involved since the early 00s) keep telling you.
At some point it might appear to be approproiiate for you to actually take
our word for it.
| submit. I will have to study how R uses autotools, because that also seems a
| bit different than I am used to.
|
| I also notice that on Debian you make a soft link of /usr/lib/R/lib/libR.so to
| /usr/lib/libR.so. Given all that has been discussed, I am unclear why.
Well noticed -- yet a stricly personaly reason via two projects I have been
(co-)authoring: littler and RInside. They both "embed" R via libR and we do
both rpath ("somewhat" verboten by Debian Policy as it hard codes a path,
hence the alternate of placing it where ldd / ldconfig find it).
But please note that that is _me_ doing this, and the R Core gospel we have
been trying for you to understand still stand: __what you insist is needed
actually is not__.
| Thank you for sticking with the thread,
Sorry for coming through as pedantic but you (and we're now at what, six
posts in and counting?) still start from the wrong (at least outside of the
gilded confines of FreeBSD) premise. It. Just. Works.
Just give us and the unknown-but-sometimes-estimated-to-be-in-the-millions of
R users some credit. What is there __works__. Seriously. Debian folks are
pendantic for technical excellence __and even they have no issue with
per-package and local shared libraries__. Which is what this is. Really.
Dirk
--
http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] shared libraries: missing soname
Martin Maechler writes: > Well, Dirk has said to have given his last reply on this thread. > I (as a member of R-core) am glad about people like Dirk who > take some of our load and helpfully answer such > questions/reports on R-devel. I am glad too. Thank you. My ultimate goal is to ensure that R works as well on FreeBSD as it does elsewhere. It mostly just works (again, appreciative), but there are a few challenges because some R conventions and some FreeBSD conventions may collide. Dirk Eddelbuettel writes: > Consider eg this: > edd@max:~$ locate Matrix.so > /usr/lib/R/library/Matrix/libs/Matrix.so > /usr/local/lib/R/site-library/Matrix/libs/Matrix.so > /usr/local/lib/R-devel/lib/R/library/Matrix/libs/Matrix.so > edd@max:~$ > I have three versions of Matrix.so. And still not problem. The third one > only enters for R-devel, not R. The first two are distinguished by R itself > _by virtue of different paths_ just like Martin and I told you. > And _no_ other program on my system knows about Matrix.so: > edd@max:~$ ldconfig -p | grep Matrix.so > edd@max:~$ ldconfig -p | wc -l > 2814 > edd@max:~$ > ldd / ldconfig do NOT know Matrix.so -- as I told you before -- despite the > fact that they know thousands of other things on this (development) machine. For a frequently-updated Debian package repository, I assume you _manually_ bump all the Debian R packages whenever the main R package is upgraded. Is that correct? That is, when a Debian user upgrades the r-base package from, say, version 3.2.5 to 3.3.1, then r-cran-tseries must/should be rebuilt/reinstalled against the new R package? If so, maybe there is something useful we could submit. I will have to study how R uses autotools, because that also seems a bit different than I am used to. I also notice that on Debian you make a soft link of /usr/lib/R/lib/libR.so to /usr/lib/libR.so. Given all that has been discussed, I am unclear why. Thank you for sticking with the thread, Joseph signature.asc Description: PGP signature __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [parallel-package] feature request: set default cluster type via environment variable
On 24/11/2016 07:30, Christian Krause wrote: Dear all, I’m working as an administrator of a High-Performance Computing (HPC) Cluster which runs on Linux. A lot of people are using R on this Linux cluster and, of course, the *parallel* package to speed up their computations. It has been our collective experience, that using |makeForkCluster| yields an overall better experience /on Linux/ than the |makePSOCKcluster|, for whatever definition of better. Let me just summarize that it works smoother. I believe, other people working with *parallel* on Linux can share this experience Usually, but not always. And the differences are mainly in initialization time, so small once workers are given a reasonable amount of work (tens of seconds each). However, as forked workers have a copy of the whole master process, forking workers can lead to excessive memory usage. Also, we did really welcome the environment variable |MC_CORES|, to be able to specify (in job submit scripts) the amount of CPU cores a user has been granted, most importantly for /dynamic resource requests/ (see below for an example). Hmm, MC_CORES is primarily for mclapply() and friends, not makeCluster(). makeForkCluster() is a 'friend' so uses it, but makePSOCKcluster() was designed for distributing across a cluster of machines (whereas makeForkCluster is restricted to a single multicore machine). What we would also appreciate - and now we finally get to the feature request - is another environment variable to choose the used cluster, as in: |export MC_CLUSTER_TYPE=FORK | Do you think something like this could be implemented in future releases? No. (Not least as 'MC_' refers to the former 'multicore' package.) PSOCK and Fork clusters are not interchangeable, and the author of the code has to check if Fork can be substituted for PSOCK (which starts with a clean R environment, and that may well be assumed). So rather, you need to ask your users to implement this in their calls to parallel::makeCluster. Parallel R job submit script This works with the Univa Grid Engine and should work with other * Grid Engine products: |#!/bin/bash # request a "parallel environment" with 2 to 20 cores #$ -pe smp 2-20 # set number of cores for the R cluster to the granted value (between 2 and 20) export MC_CORES=$NSLOTS # we want this: export MC_CLUSTER_TYPE=FORK Rscript /path/to/script.R | Best Regards -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] [parallel-package] feature request: set default cluster type via environment variable
Dear all, I’m working as an administrator of a High-Performance Computing (HPC) Cluster which runs on Linux. A lot of people are using R on this Linux cluster and, of course, the *parallel* package to speed up their computations. It has been our collective experience, that using |makeForkCluster| yields an overall better experience /on Linux/ than the |makePSOCKcluster|, for whatever definition of better. Let me just summarize that it works smoother. I believe, other people working with *parallel* on Linux can share this experience Also, we did really welcome the environment variable |MC_CORES|, to be able to specify (in job submit scripts) the amount of CPU cores a user has been granted, most importantly for /dynamic resource requests/ (see below for an example). What we would also appreciate - and now we finally get to the feature request - is another environment variable to choose the used cluster, as in: |export MC_CLUSTER_TYPE=FORK | Do you think something like this could be implemented in future releases? Parallel R job submit script This works with the Univa Grid Engine and should work with other * Grid Engine products: |#!/bin/bash # request a "parallel environment" with 2 to 20 cores #$ -pe smp 2-20 # set number of cores for the R cluster to the granted value (between 2 and 20) export MC_CORES=$NSLOTS # we want this: export MC_CLUSTER_TYPE=FORK Rscript /path/to/script.R | Best Regards -- Christian Krause Scientific Computing Administration and Support -- Phone: +49 341 97 33144 Email: christian.kra...@idiv.de -- German Centre for Integrative Biodiversity Research (iDiv) Halle-Jena-Leipzig Deutscher Platz 5e 04103 Leipzig Germany -- iDiv is a research centre of the DFG – Deutsche Forschungsgemeinschaft iDiv ist eine zentrale Einrichtung der Universität Leipzig im Sinne des § 92 Abs. 1 SächsHSFG und wird zusammen mit der Martin-Luther-Universität Halle-Wittenberg und der Friedrich-Schiller-Universität Jena betrieben sowie in Kooperation mit dem
