Re: [R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
On 9 August 2023 at 21:59, Thomas Petzoldt wrote:
| On 09.08.2023 at 19:18 Dirk Eddelbuettel wrote:
| > There is a fifth option I just realized.
| >
| > I use a variant of that to keep for example an old version of roxygen2 I
| > prefer available. Recall that R packages are generally relocatable. So
| > install the 58 you need, then do, say `mkdir /opt/shiny-r-lib` and copy them
| > in there. Setup your shiny deployment with whatever R_LIBS_USER or
| > R_LIBS_SITE works for, or simply add `.libPaths("/opt/shiny-r-lib")`.
|
| Awesome! I remember that I saw this idea somewhere at a webpage, but
| part of the explanation confused me and I forgot about it. Thanks for
| the reminder -- and your trustworthy second opinion.
|
| > Now those packages will be found first, used by Shiny and cannot be touched
| > by install.packages (be it with `apt` via `r2u` or not). Easy peasy "poor
| > man's virtual environemnt".
|
| ... and it allows multiple package sets in parallel for testing, ... or
| different .libPaths() for specific apps. The typical library is small
| (approx. 250 pkgs), so that space is no problem and a full copy took
| just a second :)
Exactly. The key is keep the default R_LIBS_* and .libPaths() for the normal
system and update, and aside versions for custom uses. After all many of us
have done just that with r-devel installed alongside r-release too.
So if only your shiny app and its setup know a separate library path, nobody
else can mess with it. In a way a more primitive and less solid form than
shielding via Docker instance, but likely "good enough".
| Many thanks, r2u and .libPaths -- should be perfect for me.
Perfect.
Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
___
R-SIG-Debian mailing list
R-SIG-Debian@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-debian
Re: [R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
On 09.08.2023 at 19:18 Dirk Eddelbuettel wrote:
Thomas,
There is a fifth option I just realized.
I use a variant of that to keep for example an old version of roxygen2 I
prefer available. Recall that R packages are generally relocatable. So
install the 58 you need, then do, say `mkdir /opt/shiny-r-lib` and copy them
in there. Setup your shiny deployment with whatever R_LIBS_USER or
R_LIBS_SITE works for, or simply add `.libPaths("/opt/shiny-r-lib")`.
Awesome! I remember that I saw this idea somewhere at a webpage, but
part of the explanation confused me and I forgot about it. Thanks for
the reminder -- and your trustworthy second opinion.
Now those packages will be found first, used by Shiny and cannot be touched
by install.packages (be it with `apt` via `r2u` or not). Easy peasy "poor
man's virtual environemnt".
... and it allows multiple package sets in parallel for testing, ... or
different .libPaths() for specific apps. The typical library is small
(approx. 250 pkgs), so that space is no problem and a full copy took
just a second :)
On 9 August 2023 at 18:58, Thomas Petzoldt wrote:
| Am 09.08.2023 um 18:17 schrieb Dirk Eddelbuettel:
| > Yes. If and when that happens, it is a bug!
|
| Not necessarily. Breaking code can also be due to an incompatible
| feature update. We know that R has both, conservative and less
| conservative universes of packages ;-) Security is another issue.
I am talking more narrowly about the failure to install a package, or getting
the wrong package, or ... as a bug in the r2u setup and provision. Those bugs
we should squash. All others continues to exist, and we fight them in
different venues.
Dirk
Agreed. I'll keep my eyes open.
Many thanks, r2u and .libPaths -- should be perfect for me.
Thomas
___
R-SIG-Debian mailing list
R-SIG-Debian@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-debian
Re: [R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
Thomas,
There is a fifth option I just realized.
I use a variant of that to keep for example an old version of roxygen2 I
prefer available. Recall that R packages are generally relocatable. So
install the 58 you need, then do, say `mkdir /opt/shiny-r-lib` and copy them
in there. Setup your shiny deployment with whatever R_LIBS_USER or
R_LIBS_SITE works for, or simply add `.libPaths("/opt/shiny-r-lib")`.
Now those packages will be found first, used by Shiny and cannot be touched
by install.packages (be it with `apt` via `r2u` or not). Easy peasy "poor
man's virtual environemnt".
On 9 August 2023 at 18:58, Thomas Petzoldt wrote:
| Am 09.08.2023 um 18:17 schrieb Dirk Eddelbuettel:
| > Yes. If and when that happens, it is a bug!
|
| Not necessarily. Breaking code can also be due to an incompatible
| feature update. We know that R has both, conservative and less
| conservative universes of packages ;-) Security is another issue.
I am talking more narrowly about the failure to install a package, or getting
the wrong package, or ... as a bug in the r2u setup and provision. Those bugs
we should squash. All others continues to exist, and we fight them in
different venues.
Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
___
R-SIG-Debian mailing list
R-SIG-Debian@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-debian
Re: [R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
Am 09.08.2023 um 18:17 schrieb Dirk Eddelbuettel:
Hi Thomas,
Thanks a lot for bringing this over from r-package-devel!
And thanks for the quick help from my side!
As for your question, "there is a lot of meat on these bones" and I really
think we should discuss this here.
Quick comments inline, more recap at the bottom.
On 9 August 2023 at 17:29, Thomas Petzoldt wrote:
| Hi,
|
| I am running a couple of shiny servers with several apps that are based
| around own CRAN packages. It worked stable for years, but due to the
| growing number of packages, the compile time for regular manual package
| installation and updates became inconvenient.
|
| Therefore, I have been very happy to use pre-compiled packages from the
| c2d4u repository -- and from now on testing r2u following a hint from
| Dirk Edelbuettel. @Dirk: installation worked like a charm.
Glad to hear!
Indeed. Just updated 58 packages and all apps continued working.
It also generally works flawlessly (and repeatedly) for myself and others in
many context involving many packages. Per the logs, we have shipped over 6.5
million packages in the ~ 15 months it has been up and generally do more than
10k each day now.
| The downside is, that the r-cran-* binaries are being installed
| automatically, together with the system update. So experienced repeated
| cases that crashed some of the shiny apps. The main reason with c2d4u
| were conflicts between the binary packages and other packages installed
| from sources. This may be less relevant with r2u, but in the interest of
| reproducibility and stability, I would prefer conservative manual updates.
Yes. If and when that happens, it is a bug!
Not necessarily. Breaking code can also be due to an incompatible
feature update. We know that R has both, conservative and less
conservative universes of packages ;-) Security is another issue.
The correct (but laborious way) is to file and report them.
Hmm, depends. I am happy to contribute bug reports, but need evidence
that it's a bug and not a feature - or configuration issue on my side.
I think the overall issue here is the mix and match from 'distro' packages,
and added packages. Given the complete coverage, I prefer r2u for
consistency. Which is why it is pinned higher at 700.
Agreed, so I applied the r2u pinning as descibed on your page.
But one can have issue if because of ("semi-random") package renmaing /
versioning / use of epoch [a leading version digit overriding] the distro
wins.
Eg we had a recent issue for r2u when following the Debian and the R 4.3.*
catchup, several packages (all around the graphics API that needed a rebuilt)
got force-rebuilt and ended up with, say, 1.2.3-2 beating 1.2.3-1.c2u2204.1.
So I quickly rebuilt that dozen. It was more of an issue for jammy (22.04)
than focal as the (distro) focal packages are generally way behind what CRAN
and hence r2u has.
The only really reliable way to fix this is by reporting the bugs the old
school way so that r2u can take care of it. I would need help from users and
that is why I am asking for it here.
| My question: what is best practise, to disallow automatic updates for
| all (or part of) r-cran-* packages? Uncommenting the complete package
| source in the apt/sources.list.d/cd4u...list file? Fiddling around with
| /etc/apt/preferences ?
|
| The ideal approach would be to put a plain textfile of all installed
| r-cran packages somewhere to the system, where packages that are to be
| upgraded (or oppositely: pinned) are just commented or outcommented.
I mentioned in the initial short reply on the other list I can see several
options (now up to four)
1) use dpkg to put packages on 'hold'. They will not be touched or
updated. Reliable, manual, tedious for many. See 'man dpkg'.
This (or same with apt-mark) is probably the easiest: Write a shell (or
R) script iterating over a text file of packages. Should work similar to
a traditional "install.packsges()"-script if almost all packages are
from r2u and only a few directly from Github.
2) look into 'apt pinning' via preferences and its wiki page at
https://wiki.debian.org/AptConfiguration etc This may work, I have not had
the patience to work the details out.
Same to me.
But there are other value beyond the
'500 to 900' default range. This may work, and be quite elegant.
3) as you hint, just 'hide' the sources.list for r2u or c2d4u but it still
risks updates from the distro versions :-/ Maybe 'easiest yet worst'.
Good point. This is what I have been afraid of.
4) maybe go the other way use eg the Rocker Project container for shiny and
run you sacred production in a shielded setting? This sounds like more work,
but it may be better. If it works for you depends.
Yes, I' just discussed a switch to docker with my colleague during
lunch. Maybe the university will install a central Shiny service in the
future or we buy the service ... For now, I am still happy with my very
simplistic
Re: [R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
Hi Thomas,
Thanks a lot for bringing this over from r-package-devel!
As for your question, "there is a lot of meat on these bones" and I really
think we should discuss this here.
Quick comments inline, more recap at the bottom.
On 9 August 2023 at 17:29, Thomas Petzoldt wrote:
| Hi,
|
| I am running a couple of shiny servers with several apps that are based
| around own CRAN packages. It worked stable for years, but due to the
| growing number of packages, the compile time for regular manual package
| installation and updates became inconvenient.
|
| Therefore, I have been very happy to use pre-compiled packages from the
| c2d4u repository -- and from now on testing r2u following a hint from
| Dirk Edelbuettel. @Dirk: installation worked like a charm.
Glad to hear!
It also generally works flawlessly (and repeatedly) for myself and others in
many context involving many packages. Per the logs, we have shipped over 6.5
million packages in the ~ 15 months it has been up and generally do more than
10k each day now.
| The downside is, that the r-cran-* binaries are being installed
| automatically, together with the system update. So experienced repeated
| cases that crashed some of the shiny apps. The main reason with c2d4u
| were conflicts between the binary packages and other packages installed
| from sources. This may be less relevant with r2u, but in the interest of
| reproducibility and stability, I would prefer conservative manual updates.
Yes. If and when that happens, it is a bug!
The correct (but laborious way) is to file and report them.
I think the overall issue here is the mix and match from 'distro' packages,
and added packages. Given the complete coverage, I prefer r2u for
consistency. Which is why it is pinned higher at 700.
But one can have issue if because of ("semi-random") package renmaing /
versioning / use of epoch [a leading version digit overriding] the distro
wins.
Eg we had a recent issue for r2u when following the Debian and the R 4.3.*
catchup, several packages (all around the graphics API that needed a rebuilt)
got force-rebuilt and ended up with, say, 1.2.3-2 beating 1.2.3-1.c2u2204.1.
So I quickly rebuilt that dozen. It was more of an issue for jammy (22.04)
than focal as the (distro) focal packages are generally way behind what CRAN
and hence r2u has.
The only really reliable way to fix this is by reporting the bugs the old
school way so that r2u can take care of it. I would need help from users and
that is why I am asking for it here.
| My question: what is best practise, to disallow automatic updates for
| all (or part of) r-cran-* packages? Uncommenting the complete package
| source in the apt/sources.list.d/cd4u...list file? Fiddling around with
| /etc/apt/preferences ?
|
| The ideal approach would be to put a plain textfile of all installed
| r-cran packages somewhere to the system, where packages that are to be
| upgraded (or oppositely: pinned) are just commented or outcommented.
I mentioned in the initial short reply on the other list I can see several
options (now up to four)
1) use dpkg to put packages on 'hold'. They will not be touched or
updated. Reliable, manual, tedious for many. See 'man dpkg'.
2) look into 'apt pinning' via preferences and its wiki page at
https://wiki.debian.org/AptConfiguration etc This may work, I have not had
the patience to work the details out. But there are other value beyond the
'500 to 900' default range. This may work, and be quite elegant.
3) as you hint, just 'hide' the sources.list for r2u or c2d4u but it still
risks updates from the distro versions :-/ Maybe 'easiest yet worst'.
4) maybe go the other way use eg the Rocker Project container for shiny and
run you sacred production in a shielded setting? This sounds like more work,
but it may be better. If it works for you depends.
Let's keep at this and see if we can make something 'already pretty good'
even better!
Cheers, Dirk
--
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
___
R-SIG-Debian mailing list
R-SIG-Debian@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-debian
[R-sig-Debian] pinning of binary r-cran-* packages from c2d4u / r2u on Ubuntu 22.04
Hi, I am running a couple of shiny servers with several apps that are based around own CRAN packages. It worked stable for years, but due to the growing number of packages, the compile time for regular manual package installation and updates became inconvenient. Therefore, I have been very happy to use pre-compiled packages from the c2d4u repository -- and from now on testing r2u following a hint from Dirk Edelbuettel. @Dirk: installation worked like a charm. The downside is, that the r-cran-* binaries are being installed automatically, together with the system update. So experienced repeated cases that crashed some of the shiny apps. The main reason with c2d4u were conflicts between the binary packages and other packages installed from sources. This may be less relevant with r2u, but in the interest of reproducibility and stability, I would prefer conservative manual updates. My question: what is best practise, to disallow automatic updates for all (or part of) r-cran-* packages? Uncommenting the complete package source in the apt/sources.list.d/cd4u...list file? Fiddling around with /etc/apt/preferences ? The ideal approach would be to put a plain textfile of all installed r-cran packages somewhere to the system, where packages that are to be upgraded (or oppositely: pinned) are just commented or outcommented. Thanks in advance, Thomas Links: https://ppa.launchpadcontent.net/c2d4u.team/c2d4u4.0+/ubuntu/ https://eddelbuettel.github.io/r2u ___ R-SIG-Debian mailing list R-SIG-Debian@r-project.org https://stat.ethz.ch/mailman/listinfo/r-sig-debian
