Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-14 Thread Martin Maechler 
> Henrik Bengtsson 
> on Sat, 14 Oct 2023 10:37:14 -0700 writes:

> On Sat, Oct 14, 2023 at 5:25 AM Martin Maechler
>  wrote:
>> 
>> > Ivan Krylov
>> > on Thu, 12 Oct 2023 18:50:30 +0300 writes:
>> 
>> > On Thu, 12 Oct 2023 11:32:24 -0400
>> > Mikael Jagan  wrote:
>> 
>> >> > mk <- file.path(R.home("share"), "make", "vars.mk")
>> >> > pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED",
>> >> > readLines(mk), value = TRUE))
>> >> > sort(strsplit(pp, " ")[[1L]])
>> >> [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
>> >> [6] "cluster""codetools"  "foreign""lattice""mgcv"
>> >> [11] "nlme"   "nnet"   "rpart"  "spatial" "survival"
>> >>
>> >> I grepped around and did not find variables in any base namespace
>> >> containing the names of these packages.  It wouldn't be too hard to
>> >> define such variables when R is configured/built, but maybe there are
>> >> "reasons" to not do that ... ?
>> 
>> > tools:::.get_standard_package_names does that at package installation
>> > time, but it's still not public API.
>> 
>> Within R-core, we have somewhat discussed this, and a few
>> minutes ago I committed a "public API" version of the above,
>> called
>> standard_package_names()
>> 
>> to R-devel (svn rev 85329), and hence probably in next year's
>> April release of R.

> Excellent. Will it be supported on all OSes?  Because, there's
> currently a source code comment saying the current implementation
> might not work on MS Windows:

> ## we cannot assume that file.path(R.home("share"), "make", "vars.mk")
> ## is installed, as it is not on Windows

Well, it does work nowadays on Windows {just checked}.
So the comment should probably be removed.

{{ I'm definitely not the expert, but AFAIK, the Windows
   installation of R has become considerably closer to the other platform
   ones, recently }}


> standard_package_names <-
> .get_standard_package_names <-
> local({
> lines <- readLines(file.path(R.home("share"), "make", "vars.mk"))
> lines <- grep("^R_PKGS_[[:upper:]]+ *=", lines, value = TRUE)
> out <- strsplit(sub("^R_PKGS_[[:upper:]]+ *= *", "", lines), " +")
> names(out) <-
> tolower(sub("^R_PKGS_([[:upper:]]+) *=.*", "\\1", lines))
> eval(substitute(function() {out}, list(out=out)), envir = topenv())
> })

> /Henrik

>> 
>> 
>> > A call to installed.packages() may take a long while because it has to
>> > list files in every library (some of which can be large and/or
>> > network-mounted) and parse each Meta/package.rds file, but at least
>> > list.files() is faster than that.
>> 
>> The above is another issue that we've wanted to improve, as some
>> of you are aware,  notably thinking about caching the result
>> .. there has been work on this during the R Sprint @ Warwick a
>> couple of weeks ago,
>> 
>> ==> https://github.com/r-devel/r-project-sprint-2023/issues/78
>> 
>> involving smart people and promising proposals (my personal view).
>> 
>> > If I had to make a choice at this point, I would hard-code the list of
>> > packages, but a better option may surface once we know what Tony needs
>> > the package lists for.
>> 
>> > --
>> > Best regards,
>> > Ivan
>> 
>> 
>> With thanks to the discussants here on R-devel,
>> and best regards,
>> Martin
>> 
>> --
>> Martin Maechler
>> ETH Zurich  and  R Core team
>> 
>> __
>> R-package-devel@r-project.org mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-14 Thread Henrik Bengtsson 
On Sat, Oct 14, 2023 at 5:25 AM Martin Maechler
 wrote:
>
> > Ivan Krylov
> > on Thu, 12 Oct 2023 18:50:30 +0300 writes:
>
> > On Thu, 12 Oct 2023 11:32:24 -0400
> > Mikael Jagan  wrote:
>
> >> > mk <- file.path(R.home("share"), "make", "vars.mk")
> >> > pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED",
> >> > readLines(mk), value = TRUE))
> >> > sort(strsplit(pp, " ")[[1L]])
> >> [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
> >> [6] "cluster""codetools"  "foreign""lattice""mgcv"
> >> [11] "nlme"   "nnet"   "rpart"  "spatial" "survival"
> >>
> >> I grepped around and did not find variables in any base namespace
> >> containing the names of these packages.  It wouldn't be too hard to
> >> define such variables when R is configured/built, but maybe there are
> >> "reasons" to not do that ... ?
>
> > tools:::.get_standard_package_names does that at package installation
> > time, but it's still not public API.
>
> Within R-core, we have somewhat discussed this, and a few
> minutes ago I committed a "public API" version of the above,
> called
>standard_package_names()
>
> to R-devel (svn rev 85329), and hence probably in next year's
> April release of R.

Excellent. Will it be supported on all OSes?  Because, there's
currently a source code comment saying the current implementation
might not work on MS Windows:

## we cannot assume that file.path(R.home("share"), "make", "vars.mk")
## is installed, as it is not on Windows
standard_package_names <-
.get_standard_package_names <-
local({
lines <- readLines(file.path(R.home("share"), "make", "vars.mk"))
lines <- grep("^R_PKGS_[[:upper:]]+ *=", lines, value = TRUE)
out <- strsplit(sub("^R_PKGS_[[:upper:]]+ *= *", "", lines), " +")
names(out) <-
tolower(sub("^R_PKGS_([[:upper:]]+) *=.*", "\\1", lines))
eval(substitute(function() {out}, list(out=out)), envir = topenv())
})

/Henrik

>
>
> > A call to installed.packages() may take a long while because it has to
> > list files in every library (some of which can be large and/or
> > network-mounted) and parse each Meta/package.rds file, but at least
> > list.files() is faster than that.
>
> The above is another issue that we've wanted to improve, as some
> of you are aware,  notably thinking about caching the result
> .. there has been work on this during the R Sprint @ Warwick a
> couple of weeks ago,
>
>  ==> https://github.com/r-devel/r-project-sprint-2023/issues/78
>
> involving smart people and promising proposals (my personal view).
>
>   > If I had to make a choice at this point, I would hard-code the list of
>   > packages, but a better option may surface once we know what Tony needs
>   > the package lists for.
>
>   > --
>   > Best regards,
>   > Ivan
>
>
> With thanks to the discussants here on R-devel,
> and best regards,
> Martin
>
> --
> Martin Maechler
> ETH Zurich  and  R Core team
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-14 Thread Martin Maechler 
> Ivan Krylov 
> on Thu, 12 Oct 2023 18:50:30 +0300 writes:

> On Thu, 12 Oct 2023 11:32:24 -0400
> Mikael Jagan  wrote:

>> > mk <- file.path(R.home("share"), "make", "vars.mk")
>> > pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED",
>> > readLines(mk), value = TRUE))
>> > sort(strsplit(pp, " ")[[1L]])  
>> [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
>> [6] "cluster""codetools"  "foreign""lattice""mgcv"
>> [11] "nlme"   "nnet"   "rpart"  "spatial" "survival"
>> 
>> I grepped around and did not find variables in any base namespace
>> containing the names of these packages.  It wouldn't be too hard to
>> define such variables when R is configured/built, but maybe there are
>> "reasons" to not do that ... ?

> tools:::.get_standard_package_names does that at package installation
> time, but it's still not public API.

Within R-core, we have somewhat discussed this, and a few
minutes ago I committed a "public API" version of the above,
called
   standard_package_names()

to R-devel (svn rev 85329), and hence probably in next year's
April release of R.


> A call to installed.packages() may take a long while because it has to
> list files in every library (some of which can be large and/or
> network-mounted) and parse each Meta/package.rds file, but at least
> list.files() is faster than that.

The above is another issue that we've wanted to improve, as some
of you are aware,  notably thinking about caching the result
.. there has been work on this during the R Sprint @ Warwick a
couple of weeks ago,

 ==> https://github.com/r-devel/r-project-sprint-2023/issues/78

involving smart people and promising proposals (my personal view).

  > If I had to make a choice at this point, I would hard-code the list of
  > packages, but a better option may surface once we know what Tony needs
  > the package lists for.

  > -- 
  > Best regards,
  > Ivan


With thanks to the discussants here on R-devel,
and best regards,
Martin

--
Martin Maechler
ETH Zurich  and  R Core team

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-12 Thread Ivan Krylov 
On Thu, 12 Oct 2023 11:32:24 -0400
Mikael Jagan  wrote:

>  > mk <- file.path(R.home("share"), "make", "vars.mk")
>  > pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED",
>  > readLines(mk), value = TRUE))
>  > sort(strsplit(pp, " ")[[1L]])  
>   [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
>   [6] "cluster""codetools"  "foreign""lattice""mgcv"
>  [11] "nlme"   "nnet"   "rpart"  "spatial" "survival"
> 
> I grepped around and did not find variables in any base namespace
> containing the names of these packages.  It wouldn't be too hard to
> define such variables when R is configured/built, but maybe there are
> "reasons" to not do that ... ?

tools:::.get_standard_package_names does that at package installation
time, but it's still not public API.

A call to installed.packages() may take a long while because it has to
list files in every library (some of which can be large and/or
network-mounted) and parse each Meta/package.rds file, but at least
list.files() is faster than that.

If I had to make a choice at this point, I would hard-code the list of
packages, but a better option may surface once we know what Tony needs
the package lists for.

-- 
Best regards,
Ivan

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-12 Thread Mikael Jagan 

Maybe something like this:

> isRecommendedPkg <- utils:::isBasePkg
> body(isRecommendedPkg)[[c(3L, 3L)]] <- "recommended"
> installed <- unique(list.files(.libPaths()))
> installed[vapply(installed, isRecommendedPkg, NA)]
 [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
 [6] "cluster""codetools"  "foreign""lattice""mgcv"
[11] "nlme"   "nnet"   "rpart"  "spatial""survival"

where in your package you would define isRecommendedPkg "manually".

Another (but quite undocumented and so maybe not "recommended" :-))
possibility is this:

> mk <- file.path(R.home("share"), "make", "vars.mk")
> pp <- sub("^.*= +", "", grep("^R_PKGS_RECOMMENDED", readLines(mk), value 
= TRUE))

> sort(strsplit(pp, " ")[[1L]])
 [1] "KernSmooth" "MASS"   "Matrix" "boot"   "class"
 [6] "cluster""codetools"  "foreign""lattice""mgcv"
[11] "nlme"   "nnet"   "rpart"  "spatial""survival"

I grepped around and did not find variables in any base namespace containing
the names of these packages.  It wouldn't be too hard to define such variables
when R is configured/built, but maybe there are "reasons" to not do that ... ?

Mikael

> It would be much faster (but slightly less reliable) to use
> list.files(.libPaths()) to get the names of all installed packages, and
> then filter them to the known list of base and recommended packages,
> which changes very rarely.
>
> Duncan Murdoch
>
> On 12/10/2023 8:34 a.m., Tony Wilkes wrote:
> > Dear all,
> >
> > In my R package that I'm developing, I use `installed.packages(priority = 
"base")` to programmatically get all core/base R packages (i.e. base, stats, 
etc.). And similarly, I use installed.packages(priority = "recommended")​` to 
programmatically get the recommended R packages (i.e. mgcv, lattice, etc.).

> >
> > However, CRAN has requested to not use `installed.packages()`, as it is 
slow. I fully get and agree with that assesment. I used installed.packages()​` 
anyway because I could not find a better, fool-proof alternative.

> >
> > Nonetheless, I was asked to change this code for optimalisation. So I would 
like to ask: how do I programmatically get all base/core R packages safely and 
efficiently, but without using `installed.packages()`? And the same question for 
recommended R packages. I have of course Googled it, and looked at R's 
documentation (though R's documentation is large, so it's easy to miss 
something); no solution found. So if any of you has a smart idea: I'm all ears.

> >
> > Thank you in advance.
> >
> > Kind regards,
> >
> > Tony.

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-12 Thread Duncan Murdoch 
It would be much faster (but slightly less reliable) to use 
list.files(.libPaths()) to get the names of all installed packages, and 
then filter them to the known list of base and recommended packages, 
which changes very rarely.


Duncan Murdoch

On 12/10/2023 8:34 a.m., Tony Wilkes wrote:

Dear all,

In my R package that I'm developing, I use `installed.packages(priority = "base")` to 
programmatically get all core/base R packages (i.e. base, stats, etc.). And similarly, I use 
installed.packages(priority = "recommended")​` to programmatically get the recommended R 
packages (i.e. mgcv, lattice, etc.).

However, CRAN has requested to not use `installed.packages()`, as it is slow. I 
fully get and agree with that assesment. I used installed.packages()​` anyway 
because I could not find a better, fool-proof alternative.

Nonetheless, I was asked to change this code for optimalisation. So I would 
like to ask: how do I programmatically get all base/core R packages safely and 
efficiently, but without using `installed.packages()`? And the same question 
for recommended R packages. I have of course Googled it, and looked at R's 
documentation (though R's documentation is large, so it's easy to miss 
something); no solution found. So if any of you has a smart idea: I'm all ears.

Thank you in advance.

Kind regards,

Tony.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel


__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

Re: [R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-12 Thread Thierry Onkelinx 
Dear Tony,

Much will depend on what information you need from the output of
installed.packages()?

Best regards,

ir. Thierry Onkelinx
Statisticus / Statistician

Vlaamse Overheid / Government of Flanders
INSTITUUT VOOR NATUUR- EN BOSONDERZOEK / RESEARCH INSTITUTE FOR NATURE AND
FOREST
Team Biometrie & Kwaliteitszorg / Team Biometrics & Quality Assurance
thierry.onkel...@inbo.be
Havenlaan 88 bus 73, 1000 Brussel
www.inbo.be

///
To call in the statistician after the experiment is done may be no more
than asking him to perform a post-mortem examination: he may be able to say
what the experiment died of. ~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data. ~ Roger Brinner
The combination of some data and an aching desire for an answer does not
ensure that a reasonable answer can be extracted from a given body of data.
~ John Tukey
///




Op do 12 okt 2023 om 14:34 schreef Tony Wilkes :

> Dear all,
>
> In my R package that I'm developing, I use `installed.packages(priority =
> "base")` to programmatically get all core/base R packages (i.e. base,
> stats, etc.). And similarly, I use installed.packages(priority =
> "recommended")​` to programmatically get the recommended R packages (i.e.
> mgcv, lattice, etc.).
>
> However, CRAN has requested to not use `installed.packages()`, as it is
> slow. I fully get and agree with that assesment. I used
> installed.packages()​` anyway because I could not find a better, fool-proof
> alternative.
>
> Nonetheless, I was asked to change this code for optimalisation. So I
> would like to ask: how do I programmatically get all base/core R packages
> safely and efficiently, but without using `installed.packages()`? And the
> same question for recommended R packages. I have of course Googled it, and
> looked at R's documentation (though R's documentation is large, so it's
> easy to miss something); no solution found. So if any of you has a smart
> idea: I'm all ears.
>
> Thank you in advance.
>
> Kind regards,
>
> Tony.
>
> [[alternative HTML version deleted]]
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel

[R-pkg-devel] Question regarding listing base and recommended packages programmatically and efficiently

2023-10-12 Thread Tony Wilkes 
Dear all,

In my R package that I'm developing, I use `installed.packages(priority = 
"base")` to programmatically get all core/base R packages (i.e. base, stats, 
etc.). And similarly, I use installed.packages(priority = "recommended")​` to 
programmatically get the recommended R packages (i.e. mgcv, lattice, etc.).

However, CRAN has requested to not use `installed.packages()`, as it is slow. I 
fully get and agree with that assesment. I used installed.packages()​` anyway 
because I could not find a better, fool-proof alternative.

Nonetheless, I was asked to change this code for optimalisation. So I would 
like to ask: how do I programmatically get all base/core R packages safely and 
efficiently, but without using `installed.packages()`? And the same question 
for recommended R packages. I have of course Googled it, and looked at R's 
documentation (though R's documentation is large, so it's easy to miss 
something); no solution found. So if any of you has a smart idea: I'm all ears.

Thank you in advance.

Kind regards,

Tony.

[[alternative HTML version deleted]]

__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel