Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Graham, On 13 July 2023 at 18:59, Graham Inggs wrote: | I believe the attached patch should do the trick. It's basically | Paul's list from message #210, plus r-cran-interval and | r-cran-maldiquant. I've also used a << relationship against the | versions in unstable, and appended a tilde at the end. I believe this | will work better for derivatives and make backports easier. Almost done building, will upload shortly. Thanks, Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Dirk
On Thu, 13 Jul 2023 at 16:25, Dirk Eddelbuettel wrote:
> Sounds good, and thanks for the assist! I should be able to provide a pretty
> quick turn-around.
I believe the attached patch should do the trick. It's basically
Paul's list from message #210, plus r-cran-interval and
r-cran-maldiquant. I've also used a << relationship against the
versions in unstable, and appended a tilde at the end. I believe this
will work better for derivatives and make backports easier.
Regards
Graham
--- a/debian/control
+++ b/debian/control
@@ -52,31 +52,17 @@
Provides: r-api-4.0, r-graphics-engine-${graphicsengine:Version}
Recommends: r-recommended, r-base-dev, r-doc-html
Suggests: elpa-ess, r-doc-info | r-doc-pdf, r-mathlib, r-base-html
-Breaks: r-bioc-graph (<< 1.62.0-1~),
-r-cran-bbmle (<< 1.0.20-5~),
-r-cran-biocmanager (<< 1.30.4+dfsg-2~),
-r-cran-caret (<< 6.0-84-2~),
-r-cran-cmprsk (<< 2.2-8-1~),
-r-cran-coin (<< 1.3-0-1~),
-r-cran-dendextend (<< 1.12.0+dfsg-1~),
-r-cran-fields (<< 9.8-3-1~),
-r-cran-filehash (<< 2.4-2-2~),
-r-cran-future (<< 1.14.0+dfsg-1~),
-r-cran-genetics (<< 1.3.8.1.2-1~),
-r-cran-haplo.stats (<< 1.7.9-4~),
-r-cran-igraph (<< 1.2.4.1-1~),
-r-cran-lava (<< 1.6.5-1~),
-r-cran-libcoin (<< 1.0-4-1~),
-r-cran-msm (<< 1.6.7-1~),
-r-cran-permute (<< 0.9-5-1~),
-r-cran-phangorn (<< 2.5.5-1~),
-r-cran-popepi (<< 0.4.7-1~),
-r-cran-recipes (<< 0.1.6-1~),
-r-cran-sp (<< 1:1.3-1-2~),
-r-cran-spam (<< 2.2-2-1~),
-r-cran-units (<< 0.6-3-1~),
-r-cran-vegan (<< 2.5-5+dfsg-1~),
-r-cran-zelig (<< 5.1.6.1-1~)
+Breaks: r-cran-cairo (<< 1.6-0-4~),
+r-cran-intervals (<< 0.15.3-1~),
+r-cran-fnn (<< 1.1.3.2-1~),
+r-cran-magick (<< 2.7.4+dfsg-2~),
+r-cran-maldiquant (<< 1.22.1-1~),
+r-cran-ps (<< 1.7.5-1~),
+r-cran-ragg (<< 1.2.5-3~),
+r-cran-svglite (<< 2.1.1-3~),
+r-cran-tibble (<< 3.2.1+dfsg-2~),
+r-cran-tikzdevice (<< 0.12.4-3~),
+r-cran-vdiffr (<< 1.0.5-3~)
Description: GNU R core of statistical computation and graphics system
R is a system for statistical computation and graphics. It consists
of a language plus a run-time environment with graphics, a debugger,
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Graham, On 13 July 2023 at 11:14, Graham Inggs wrote: | On Wed, 12 Jul 2023 at 19:07, Dirk Eddelbuettel wrote: | > On 12 July 2023 at 19:47, Paul Gevers wrote: | > | Yes, you only need to carry the Breaks until in the next release. So | > | every Breaks that's present in the r-base package in bookworm can be | > | removed from the r-base package in unstable. | > | > Good point and much less ugly then :) | | I'll prepare a patch dropping the old and adding the new Breaks. Sounds good, and thanks for the assist! I should be able to provide a pretty quick turn-around. Onwards! Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Dirk On Wed, 12 Jul 2023 at 19:07, Dirk Eddelbuettel wrote: > On 12 July 2023 at 19:47, Paul Gevers wrote: > | On 12-07-2023 16:02, Dirk Eddelbuettel wrote: > | > I can add the Breaks as a 'best of the worse alternative'. And, I > presume, I > | > can remove the existing four-year breaks? [1] > | > | Yes, you only need to carry the Breaks until in the next release. So > | every Breaks that's present in the r-base package in bookworm can be > | removed from the r-base package in unstable. > > Good point and much less ugly then :) I'll prepare a patch dropping the old and adding the new Breaks. Regards Graham
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Paul, On 12 July 2023 at 19:47, Paul Gevers wrote: | On 12-07-2023 16:02, Dirk Eddelbuettel wrote: | > I can add the Breaks as a 'best of the worse alternative'. And, I presume, I | > can remove the existing four-year breaks? [1] | | Yes, you only need to carry the Breaks until in the next release. So | every Breaks that's present in the r-base package in bookworm can be | removed from the r-base package in unstable. Good point and much less ugly then :) Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi, On 12-07-2023 16:02, Dirk Eddelbuettel wrote: I can add the Breaks as a 'best of the worse alternative'. And, I presume, I can remove the existing four-year breaks? [1] Yes, you only need to carry the Breaks until in the next release. So every Breaks that's present in the r-base package in bookworm can be removed from the r-base package in unstable. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Paul, On 11 July 2023 at 20:36, Paul Gevers wrote: | On 11-07-2023 02:43, Dirk Eddelbuettel wrote: | I'm totally on board for technical excellence, although I think we have | different things in mind when we say that. | | In Debian, with more QA than we ever had before, we're finding a class | of issues that often went unnoticed years ago. One of these things is Of course I am not against testing auto-upgrades. I imagine nobody is. What I am not happy about is that we fell into a hole we would not need to be in, ideally. Please hear me out on this: - R has an annual release cycle at the end of April - During that cycle the 'next' release (in VCS) is referred to as r-devel. - CRAN checks all packages at all uploads, as well as periodically, against r-devel. - When a change is needed because something would break once r-devel becomes 'r-release' they contact the package author and request the change, this is a hard requirement and non-compliant package are thrown off CRAN. No - breakage allowed! - What I showed with maldiquant was that its upstream made the change in March still well before the R 4.3.0 release requiring it - So we could have (we had the freeze, more below) had the package which would have passed both 'r-release' then and 'r-devel then' (which is 'r-release now') and everything would pass. Even under our tests. That is an ideal I would really like us to move towards with the CRAN packages in Debian. As CRAN makes it so easy for us to 'always build / build @HEAD' we really should take the fullest advantage of it. So I typically roll up my packages the day or week of a CRAN change. (And maintain 2 x 21k binaries off CRAN in r2u, but that is a different story.) As it happens, not everybody does, sometimes we have freezes, and other things happen. Also the number of CRAN packages increases of course and the net-net of that is that we then have to spend precious manual time picking up the pieces. Clearly not ideal, but better than having installation bugs. | that partial upgrades can leave you in a bad state. So more and more we | see that packages "have to" add Breaks to tell apt that when you want to | upgrade package X, you also have to upgrade package Y if you happen to | have that installed. As package Y can not tell that, package X has to | add the Breaks on the broken version of Y. As an example, see the list | of Breaks in libc6 [1]. While partial upgrades aren't officially Thanks for that. An impressively long list! | supported, we rely on them nevertheless (even if only for QA (piuparts, | autopkgtest)), and as a Release Team member I consider that class of | fixes technical excellence: ensuring as best as we can that the user | that upgrades a package keeps a working system. Yes. | While a rebuild of everything combined with bumping the "api" would | achieve that, I'm much more in favor of targeted Breaks, like we have | been discussing here. It's typically more work, but it's more correct. Fully agreed. | For the future, with the recent change in dh-r, r-base will be much less | impacted by this "problem" as the new uploads of reverse dependencies | can migrate *before* r-base, and hence this class of issues will I don't think so. The recent change helps with the (approximately a handful or two CRAN packages) setting the graphics engine check (out of a total of maybe 1300 CRAN/BioC packages at Debian leaving the many others unaffected). | disappear once that happens (autopkgtest failures are retried after a | day). So unless somebody investigates the issues in time, the retry will We will get the same breakage next time will CRAN assumes (and ensures !!) everything is current at HEAD, we usually have slippage for various reasons so in practice I fear ... here we are and remain. | pass after the migration and the issue will no longer block r-base. I | can live with that, but I find it a shame nevertheless. Yep. We should aim to have 'less shame, more technical excellence'. | So, what do you say: technical excellence and you add the Breaks? Or we | let this slip in? I prefer the former, I can live with the latter. I can add the Breaks as a 'best of the worse alternative'. And, I presume, I can remove the existing four-year breaks? [1] Cheers, Dirk [1] edd@rob:~/deb/r-base(master)$ git blame debian/control | grep -A24 Breaks 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 55) Breaks: r-bioc-graph (<< 1.62.0-1~), 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 56) r-cran-bbmle (<< 1.0.20-5~), 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 57) r-cran-biocmanager (<< 1.30.4+dfsg-2~), 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 58) r-cran-caret (<< 6.0-84-2~), 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 59) r-cran-cmprsk (<< 2.2-8-1~), 52de7d776d (Dirk Eddelbuettel 2019-08-08 19:11:55 -0500 60) r-cran-coin (<<
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Dirk, On 11-07-2023 02:43, Dirk Eddelbuettel wrote: I still have hopes that we can let technical excellence rule and not require blunt instruments such as forced recompilation. I'm totally on board for technical excellence, although I think we have different things in mind when we say that. In Debian, with more QA than we ever had before, we're finding a class of issues that often went unnoticed years ago. One of these things is that partial upgrades can leave you in a bad state. So more and more we see that packages "have to" add Breaks to tell apt that when you want to upgrade package X, you also have to upgrade package Y if you happen to have that installed. As package Y can not tell that, package X has to add the Breaks on the broken version of Y. As an example, see the list of Breaks in libc6 [1]. While partial upgrades aren't officially supported, we rely on them nevertheless (even if only for QA (piuparts, autopkgtest)), and as a Release Team member I consider that class of fixes technical excellence: ensuring as best as we can that the user that upgrades a package keeps a working system. While a rebuild of everything combined with bumping the "api" would achieve that, I'm much more in favor of targeted Breaks, like we have been discussing here. It's typically more work, but it's more correct. For the future, with the recent change in dh-r, r-base will be much less impacted by this "problem" as the new uploads of reverse dependencies can migrate *before* r-base, and hence this class of issues will disappear once that happens (autopkgtest failures are retried after a day). So unless somebody investigates the issues in time, the retry will pass after the migration and the issue will no longer block r-base. I can live with that, but I find it a shame nevertheless. So, what do you say: technical excellence and you add the Breaks? Or we let this slip in? I prefer the former, I can live with the latter. Paul [1] https://tracker.debian.org/media/packages/g/glibc/control-2.37-5 OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
On 10 July 2023 at 19:43, Dirk Eddelbuettel wrote: | Someone simply didn't update our Debian package, so it lacks this change and | fingers point at r-base when the fault, if there is one, is to let our | package slip behind a compilation and code standard established at CRAN for | the R 4.3.0 relese in April. | | I still have hopes that we can let technical excellence rule and not require | blunt instruments such as forced recompilation. | | Because with slips like this, even forcing recompilation of over 1000+ | sources packages times 10+ architectures (for binary-any) will not help | against stale code, and hence mismatched expectations in the language engine | (r-base) and its client packages. I should have double-checked. 1.22.1-1 is in unstable, so that is good, and hence works with R 4.3.[01]. So what tripped me up is the (known, and expected, if "you know") failure in the (hence not all that informative) autopkgtest of trying R 4.3.1 with the outdated maldiquant 1.22 (not 1.22.1). I am not versed well-enough in the mechanics of release details. If the only way to get r-base into testing consists of having a Breaks for r-cran-maldiquant (<< 1.22.1) then I guess that is what we need to do. Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Paul, Here is a case in point from looking at the current excluses list (which is by now indeed a little shorter). One package that jumps out is r-cran-maldiquant. We are at version 1.22, with Debian build 1.22-1. But one second at the CRAN site and the page for the package shows that it is upstream at 1.22.1, since March, with a sole entry in NEWS being CHANGES IN MALDIquant VERSION 1.22.1 [2023-03-20]: -- * Use symbols instead of names in `.Call` for faster lookup of C functions. So there it is. Someone simply didn't update our Debian package, so it lacks this change and fingers point at r-base when the fault, if there is one, is to let our package slip behind a compilation and code standard established at CRAN for the R 4.3.0 relese in April. I still have hopes that we can let technical excellence rule and not require blunt instruments such as forced recompilation. Because with slips like this, even forcing recompilation of over 1000+ sources packages times 10+ architectures (for binary-any) will not help against stale code, and hence mismatched expectations in the language engine (r-base) and its client packages. Best, Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Dirk, On 09-07-2023 21:09, Dirk Eddelbuettel wrote: So this is where R 4.3.[01] will possibly break with some older packages. But the fix is simple because when R 4.3.0 came out all packages at CRAN complied. We need to have current packages that correspond to the R 4.3.0 standard. (If one were super picky one could call this an ABI/API change and reason for forcing _all_ packages to be rebuild. I never advocated for that but I am getting tired of this process. But we need to throw that bomb at some point just say 'fsck it' and rebuild _all_ packages. Feels like overkill but so is wasting weeks on this. Reading this a second time, I think your proposal is to acknowledge that r-base sometimes introduces unanticipated changes, and if we want to be on the safe side, we should rebuild everything when the minor version bumps. I guess that's more or less (I'm not sure how different r is in that respect) what other interpretation based languages like python, ruby, php and perl are doing too. r-base already has a "handle" for that: r-api-*. We *could* decide to just bump that on every minor bump of r-base and that would mean a transition every time that happens (mostly like those other languages except ruby, python and php also have a "defaults" package to explicitly switch from one version to the next), but at least our tools would help there. Apart from the potential overkill, the major (current) drawback is that for arch:all binaries, we still need source-full uploads as binNMU's don't work for them. Is that what you are proposing? (For next time maybe?) Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi, On 09-07-2023 21:09, Dirk Eddelbuettel wrote: | I don't understand this then. For several packages we're seeing failures | in testing even if we only use r-base from unstable and everything else | from testing to run the test. They pass with a rebuild r-cran-fnn and/or | a rebuild and updated r-cran-ps, and/or r-cran-tibble. (Sorry, in my | previous message I think I added r-cran-dplyr by mistake, misremembered). It would be useful to break this down into concrete reproducible examples. Several in the set that I explicitly tested: r-cran-stars (needs r-cran-tibble and r-cran-interval) r-cran-spacetime (needs r-cran-interval) r-cran-uwot (needs r-cran-fnn) r-cran-xopen (needs r-cran-ps) So this is where R 4.3.[01] will possibly break with some older packages. But the fix is simple because when R 4.3.0 came out all packages at CRAN complied. We need to have current packages that correspond to the R 4.3.0 standard. And Andreas already ensured (nearly) everything is rebuild by now, so that part is done. To be clear, I think I understand it when you say this is not caused by r-base, but rather by those packages that need rebuilding with the new r-base. However, to ensure those packages are upgraded alongside r-base, r-base needs to force that. And that's why there is the Breaks. I think the list is now: r-cran-cairo (<= 1.6-0-1) r-cran-fnn (<= 1.1.3.1-1) r-cran-magick (<= 2.7.3+dfsg-3) r-cran-ps (<= 1.7.2-1) r-cran-ragg (<= 1.2.5-1) r-cran-svglite (<= 2.1.1-1) r-cran-tibble (<= 3.1.8+dfsg-1) r-cran-tikzdevice (<= 0.12.4-1) r-cran-vdiffr (<= 1.0.5-1) they built with. My point is that the accept-vs-break comes from the package, not from R.) Sure, but the package in testing and stable can't (easily) tell that anymore in their relationships as they are already shipped, so packages moving into testing should solve it. I think only r-base is in the position to add the right Breaks. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Paul,
On 9 July 2023 at 20:11, Paul Gevers wrote:
| On 09-07-2023 18:41, Dirk Eddelbuettel wrote:
| > On 9 July 2023 at 17:40, Paul Gevers wrote:
| > | Did we already discuss that r-cran-ps also seems to be impacted by the
| > | r-base change of the symbols thingy, as can be seen in r-cran-xopen [1].
| >
| > Correct me if I am wrong but the "symbols thingy" was not a change in R
4.2.*
| > to R 4.3.*. It was a change by some packages opting into different behavior.
|
| I don't understand this then. For several packages we're seeing failures
| in testing even if we only use r-base from unstable and everything else
| from testing to run the test. They pass with a rebuild r-cran-fnn and/or
| a rebuild and updated r-cran-ps, and/or r-cran-tibble. (Sorry, in my
| previous message I think I added r-cran-dplyr by mistake, misremembered).
It would be useful to break this down into concrete reproducible examples.
| > | 33s 10. ps:::not_null(.Call("psll_connections", p))
| >
| > Tis would be a bug in r-cran-ps and I think a Breaks may be warranted.
|
| Ok, let's remember this.
|
| > Given
| > that ps always had 'native symbols', maybe testthat changed?
|
| But I think (I would need to check for the autopkgtest fallback) in none
| of the tests, the version of testthat in testing changed between passing
| and failing tests. Would testthat embed something during the build of
| the binaries (from the name, I would assume not)?
I don't think it would do anything _explicit_.
I think what we are seeing is simply 'fragility from some packages with
larger tails'. If you install 'testthat' (presumably just to run tests) you
end up with with 30+ packages loaded (without considering Suggests:). It is
similar with other 'tidyverse' packages (dplyr, tibble, vctrs, ...) are all
part of that.
| > | Same for r-cran-fnn, which impacts r-cran-uwof [2].
I just looked at FNN, it is very narrow package at its core, it gets a bit of
tail via the Suggests of chemometrics.
| > | I think what we should do is add a versioned Breaks in r-base on
| > | , r-cran-ps, r-cran-tibble,
| > | r-cran-dplyr, r-cran-fnn. I think that's the right thing to do for
| >
| > I think it would be best to work out to corresponding package pairings and
| > apply the Breaks to them. If we can.
|
| Sure, lets add the Breaks to the place where it belongs.
|
| > For spacetime and stars I suspect (based on past experience) possible
| > interaction from the underlying graphics libraries.
|
| Good to hear, didn't check yet, but will shortly (geospatial).
It's a complex block. spacetime is one of the older ones using 'sp' (and then
'raster' via Suggests), 'stars' is newer using 'sf'. Sometimes these prefer /
work better with a consistent rebuild.
| > If I am not mistaken all of these were already in the Excuses list before we
| > made addition of the r-graphics-engine-* tag (which was taken care of for R
| > 4.3.* already, having it may help if another change happens like that).
|
| Sure, I'd hope the r-graphics-engine-* Provides only reduced issue, so
| I'm currently considering them to be different. But I might be proven
| wrong easily.
I don't think it had a net effect this. The
|
| > Unfortunately I find
| > the reports a little hard to read and am hence not very efficient at
| > pinpointing underlying causes. Above you pointed eg at [2] for uwot, I see
no
| > error in there :-/
|
| Well, that log has two tests. The first second one passes, the first one
| has:
| 51s > library(testthat)
| 51s > library(uwot)
| 51s Loading required package: Matrix
| 53s >
| 53s > test_check("uwot")
| 53s Error in FNN::get.knn(X, k) : DLL requires the use of native symbols
| 53s Calls: test_check ... eval -> create_data -> find_nn -> FNN_nn ->
|
| 53s Execution halted
But uwot itself does not force symbols:
https://github.com/jlmelville/uwot/blob/master/src/RcppExports.cpp#L228-L231
and I mentioned, using just those two lines in common. So the forceSymbols
comes from somewhere else.
Ok, I rechecked. R 4.3.0 has this
* Attempting to use a character string naming a foreign function
entry point in a foreign function call in a package will now
signal an error if the packages has called R_forceSymbols to
specify that symbols must be used.
It used to _ignore_ the combinatation of calling R_forceSymbols _and_ use of
non-symbols / quoted identifiers. Now it errors: if you have R_forceSymbols
each .Call() will require symbols (not strings).
So this is where R 4.3.[01] will possibly break with some older packages.
But the fix is simple because when R 4.3.0 came out all packages at CRAN
complied. We need to have current packages that correspond to the R 4.3.0
standard.
(If one were super picky one could call this an ABI/API change and reason for
forcing _all_ packages to be rebuild. I never advocated for that but I am
getting tired of this process. But we need to throw that bomb at some point
just
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi Dirk,
Thanks.
On 09-07-2023 18:41, Dirk Eddelbuettel wrote:
On 9 July 2023 at 17:40, Paul Gevers wrote:
| Did we already discuss that r-cran-ps also seems to be impacted by the
| r-base change of the symbols thingy, as can be seen in r-cran-xopen [1].
Correct me if I am wrong but the "symbols thingy" was not a change in R 4.2.*
to R 4.3.*. It was a change by some packages opting into different behavior.
I don't understand this then. For several packages we're seeing failures
in testing even if we only use r-base from unstable and everything else
from testing to run the test. They pass with a rebuild r-cran-fnn and/or
a rebuild and updated r-cran-ps, and/or r-cran-tibble. (Sorry, in my
previous message I think I added r-cran-dplyr by mistake, misremembered).
| 33s 10. ps:::not_null(.Call("psll_connections", p))
Tis would be a bug in r-cran-ps and I think a Breaks may be warranted.
Ok, let's remember this.
Given
that ps always had 'native symbols', maybe testthat changed?
But I think (I would need to check for the autopkgtest fallback) in none
of the tests, the version of testthat in testing changed between passing
and failing tests. Would testthat embed something during the build of
the binaries (from the name, I would assume not)?
| Same for r-cran-fnn, which impacts r-cran-uwof [2].
|
| I think what we should do is add a versioned Breaks in r-base on
| , r-cran-ps, r-cran-tibble,
| r-cran-dplyr, r-cran-fnn. I think that's the right thing to do for
I think it would be best to work out to corresponding package pairings and
apply the Breaks to them. If we can.
Sure, lets add the Breaks to the place where it belongs.
For spacetime and stars I suspect (based on past experience) possible
interaction from the underlying graphics libraries.
Good to hear, didn't check yet, but will shortly (geospatial).
If I am not mistaken all of these were already in the Excuses list before we
made addition of the r-graphics-engine-* tag (which was taken care of for R
4.3.* already, having it may help if another change happens like that).
Sure, I'd hope the r-graphics-engine-* Provides only reduced issue, so
I'm currently considering them to be different. But I might be proven
wrong easily.
Unfortunately I find
the reports a little hard to read and am hence not very efficient at
pinpointing underlying causes. Above you pointed eg at [2] for uwot, I see no
error in there :-/
Well, that log has two tests. The first second one passes, the first one
has:
51s > library(testthat)
51s > library(uwot)
51s Loading required package: Matrix
53s >
53s > test_check("uwot")
53s Error in FNN::get.knn(X, k) : DLL requires the use of native symbols
53s Calls: test_check ... eval -> create_data -> find_nn -> FNN_nn ->
53s Execution halted
Obviously, I too want my package r-base in testing and I will help. But I
feel that pinning a large Breaks list on it seems a little inefficient /
unfair if the package was not causing the change. We can do if there is (as
we r-graphics-engine-*) an overwhelming feel "that we must".
I want the Breaks in the right location. If we locate a more logical
location than r-base, that's where it should go. At least the packages
involved in the r-graphics-engine would do nice, as it's really the
change in r-base that broke them (will not be needed anymore after the
release of trixie as now we have the Provides): r-cran-cairo,
r-cran-magick, r-cran-ragg, r-cran-svglite, r-cran-tikzdevice and
r-cran-vdiffr.
Paul
OpenPGP_signature
Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
On 9 July 2023 at 11:41, Dirk Eddelbuettel wrote: | For spacetime and stars I suspect (based on past experience) possible | interaction from the underlying graphics libraries. Absent-minded typing error: "geospatial", of course. Not "graphics". Dirk -- dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Paul,
On 9 July 2023 at 17:40, Paul Gevers wrote:
| Did we already discuss that r-cran-ps also seems to be impacted by the
| r-base change of the symbols thingy, as can be seen in r-cran-xopen [1].
Correct me if I am wrong but the "symbols thingy" was not a change in R 4.2.*
to R 4.3.*. It was a change by some packages opting into different behavior.
| In unstable r-cran-xopen works. If I take r-cran-ps, r-cran-xopen and
| r-base from unstable and test in testing, r-cran-xopen works. If I only
| take r-base and r-cran-ps from unstable and test in testing,
| r-cran-xopen works. Can somebody with R understanding confirm?
|
| 33s Error in `not_null(.Call("psll_connections", p))`: DLL requires
| the use of native symbols
| 33s Backtrace:
| 33s 1. testthat::test_that(...)
| 33sat test.R:4:0
| 33s 2. testthat:::test_code(desc, code, env = parent.frame(),
| reporter = reporter)
| 33s 3. reporter$start_test(context = reporter$.context, test = test)
| 33s 4. testthat:::o_apply(self$reporters, "start_test", context, test)
| 33s 5. base::lapply(objects, f)
| 33s 6. testthat (local) FUN(X[[i]], ...)
| 33s 7. x$start_test(...)
| 33s 8. ps::ps_connections(ps_handle())
| 33s 9. ps:::psl_connections(p)
| 33s 10. ps:::not_null(.Call("psll_connections", p))
Tis would be a bug in r-cran-ps and I think a Breaks may be warranted. We
are apparently between version 1.7.2 and 1.7.5 of package (r-cran-)ps but I
see no smoking gun in https://github.com/r-lib/ps/blob/main/NEWS.md that
would have caused this.
Looking further, `git blame` indicates that the package had
`registation=TRUE` for five years / all its existence, see line 2 here
https://github.com/r-lib/ps/blame/357f9c01f5db95b67ce9e230282391bc835afca4/R/ps.R
It also used 'forceSymbols' for the same five years (lines 99 to 101 here
https://github.com/r-lib/ps/blame/357f9c01f5db95b67ce9e230282391bc835afca4/src/init.c
So I think the issue here may not be coming from ps. I has not changed how it
refers to its symbols. Same R API, same usage.
As the last entry here is into the (unexported) ps:::not_null we can look
into it. It just indexes with map_lgl which is a local function (from R/utils)
and calls psll_connection, a local compiled function in the package. Given
that ps always had 'native symbols', maybe testthat changed?
However, it does not force symbols :-/ Lines 20 and 21 use the two required
initialization but not the optional symbol forcing that eg ps has.
https://github.com/r-lib/testthat/blob/main/src/init.c
So I got nothing here. Cause is unclear to me.
| Same for r-cran-fnn, which impacts r-cran-uwof [2].
|
| I think what we should do is add a versioned Breaks in r-base on
| , r-cran-ps, r-cran-tibble,
| r-cran-dplyr, r-cran-fnn. I think that's the right thing to do for
I think it would be best to work out to corresponding package pairings and
apply the Breaks to them. If we can.
| bookworm to trixie upgrades (and current trixie to trixie with the new
| r-base). Has anyone see other packages throwing that "DLL requires the
| use of native symbols" error? I spotted the ones below [3, 4, 5, 6], but
| I haven't identified which package brings in the issue. I first thought
| it would be from the package itself, but for some (r-cran-spacetime and
| r-cran-stars) their versions in unstable fail their own testsuite in
For spacetime and stars I suspect (based on past experience) possible
interaction from the underlying graphics libraries.
| testing. Would it hint at the same problem for the packages, or just for
| their tests? I suspect the former, then they should also need to be
| added to the Breaks list.
|
| Paul
|
| [1]
|
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-xopen/35575366/log.gz
| [2]
|
https://ci.debian.net/data/autopkgtest/testing/i386/r/r-cran-uwot/35590669/log.gz
| [3]
|
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-intervals/35575046/log.gz
| [4]
|
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-maldiquant/35575320/log.gz
| [5]
|
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-spacetime/35575371/log.gz
| [6]
|
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-stars/35583884/log.gz
If I am not mistaken all of these were already in the Excuses list before we
made addition of the r-graphics-engine-* tag (which was taken care of for R
4.3.* already, having it may help if another change happens like that).
So it short, the vast majority of R packages is now fine. A persistent
subset is not under the testing regime of autopkgtest. Unfortunately I find
the reports a little hard to read and am hence not very efficient at
pinpointing underlying causes. Above you pointed eg at [2] for uwot, I see no
error in there :-/
Obviously, I too want my package r-base in testing and I will help. But I
feel that pinning a large Breaks list on it seems a little inefficient /
unfair if
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi,
On 09-07-2023 16:20, Andreas Tille wrote:
I'm working my through the list and the ppc64el ci workers have a bit of
backlog; we're getting somewhere, but I'm think I'm still also seeing
different failure modes than the graphics engine, tibble and dplyr.
I admit the only chance I personally see to clarify this question is to
open an issue at the tibble git repository[1]. May be we also need
something like an r-cran-tibble-api?
Did we already discuss that r-cran-ps also seems to be impacted by the
r-base change of the symbols thingy, as can be seen in r-cran-xopen [1].
In unstable r-cran-xopen works. If I take r-cran-ps, r-cran-xopen and
r-base from unstable and test in testing, r-cran-xopen works. If I only
take r-base and r-cran-ps from unstable and test in testing,
r-cran-xopen works. Can somebody with R understanding confirm?
33s Error in `not_null(.Call("psll_connections", p))`: DLL requires
the use of native symbols
33s Backtrace:
33s 1. testthat::test_that(...)
33sat test.R:4:0
33s 2. testthat:::test_code(desc, code, env = parent.frame(),
reporter = reporter)
33s 3. reporter$start_test(context = reporter$.context, test = test)
33s 4. testthat:::o_apply(self$reporters, "start_test", context, test)
33s 5. base::lapply(objects, f)
33s 6. testthat (local) FUN(X[[i]], ...)
33s 7. x$start_test(...)
33s 8. ps::ps_connections(ps_handle())
33s 9. ps:::psl_connections(p)
33s 10. ps:::not_null(.Call("psll_connections", p))
Same for r-cran-fnn, which impacts r-cran-uwof [2].
I think what we should do is add a versioned Breaks in r-base on
, r-cran-ps, r-cran-tibble,
r-cran-dplyr, r-cran-fnn. I think that's the right thing to do for
bookworm to trixie upgrades (and current trixie to trixie with the new
r-base). Has anyone see other packages throwing that "DLL requires the
use of native symbols" error? I spotted the ones below [3, 4, 5, 6], but
I haven't identified which package brings in the issue. I first thought
it would be from the package itself, but for some (r-cran-spacetime and
r-cran-stars) their versions in unstable fail their own testsuite in
testing. Would it hint at the same problem for the packages, or just for
their tests? I suspect the former, then they should also need to be
added to the Breaks list.
Paul
[1]
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-xopen/35575366/log.gz
[2]
https://ci.debian.net/data/autopkgtest/testing/i386/r/r-cran-uwot/35590669/log.gz
[3]
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-intervals/35575046/log.gz
[4]
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-maldiquant/35575320/log.gz
[5]
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-spacetime/35575371/log.gz
[6]
https://ci.debian.net/data/autopkgtest/testing/amd64/r/r-cran-stars/35583884/log.gz
OpenPGP_signature
Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Am Sat, Jul 08, 2023 at 10:35:17PM +0200 schrieb Paul Gevers: > Indeed, I think the pattern is that if we test in testing, with r-cran from > unstable and r-cran-tibble from testing it fails, but with r-cran from > unstable and r-cran-tibble from unstable, it works. > > I'm working my through the list and the ppc64el ci workers have a bit of > backlog; we're getting somewhere, but I'm think I'm still also seeing > different failure modes than the graphics engine, tibble and dplyr. I admit the only chance I personally see to clarify this question is to open an issue at the tibble git repository[1]. May be we also need something like an r-cran-tibble-api? Kind regards Andreas. [1] https://github.com/tidyverse/tibble -- http://fam-tille.de
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi, On 06-07-2023 21:18, Andreas Tille wrote: Am Thu, Jul 06, 2023 at 08:28:45PM +0200 schrieb Paul Gevers: On 06-07-2023 19:08, Paul Gevers wrote: I'm seeing in several tests where things seem to work when r-cran-tibble from unstable is involved and fail if the version from unstable is used; Are you sure there is no typo in your sentence? At least I fail to understand. I assume the latter "unstable" should be "testing", right? Indeed, I think the pattern is that if we test in testing, with r-cran from unstable and r-cran-tibble from testing it fails, but with r-cran from unstable and r-cran-tibble from unstable, it works. I'm working my through the list and the ppc64el ci workers have a bit of backlog; we're getting somewhere, but I'm think I'm still also seeing different failure modes than the graphics engine, tibble and dplyr. Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#1040001: Role of tibble? (Was: Bug#1040001: Seeking advise how to proceed with the transition / move R stack to testing)
Hi,
Am Thu, Jul 06, 2023 at 08:28:45PM +0200 schrieb Paul Gevers:
> On 06-07-2023 19:08, Paul Gevers wrote:
> > Yes, we'll take care of that where it looks sane to do so (examples of
> > that are r-cran-epi); I'll manually schedule the "combined" tests, such
> > that they disappear from the excuses if they then pass.
>
> I'm seeing in several tests where things seem to work when r-cran-tibble
> from unstable is involved and fail if the version from unstable is used;
Are you sure there is no typo in your sentence? At least I fail to
understand. I assume the latter "unstable" should be "testing", right?
> e.g. here: https://ci.debian.net/packages/r/r-cran-rsample/testing/amd64/
I can only say that tibble is used by a lot of packages directly as
dependency (69 in my locally stored repositories - may be some more).
In addition there are further dependencies of higher order.
> Is there something special about r-cran-tibble? It didn't grow a dependency
> on r-graphics-engine according to
> https://buildd.debian.org/status/fetch.php?pkg=r-cran-tibble=amd64=3.2.1%2Bdfsg-2=1688629916=0
> so it doesn't seem to be involved in that part of the discussion.
I confirm it shouldn't be involved into the r-graphics-engine issue.
May be there is some other "hidden" transition needed which is connected
to some interface of tibble? Dirk, can you shed some light into this?
Kind regards
Andreas.
--
http://fam-tille.de
