Thanks, Simon and Uwe, for your inputs.
It's not that TMB requires a particular version of Matrix, it just
suggests using the same version of Matrix as it was built with. We may
have seen this problem before, with innocuous changes in Rcpp breaking
downstream packages.
A different example, running in two separate sessions for comparison, is
shown below.
Windows may be rebuilding more often than strictly necessary, but it
seems the right thing to do to me. We could assess the overhead using
historic data to see how big of an impact that would be.
I hear you're proposing that the TMB maintainers release an updated
version whenever Matrix is updated?
Best regards
Kirill
# Example 1: happy path
dir.create("templib")
.libPaths(normalizePath("templib"))
Sys.setenv(R_LIBS_USER = .libPaths()[[1]])
# Sanity check
rownames(installed.packages())
#> [1] "base" "boot" "class" "cluster" "codetools"
#> [6] "compiler" "datasets" "foreign" "graphics" "grDevices"
#> [11] "grid" "KernSmooth" "lattice" "MASS" "Matrix"
#> [16] "methods" "mgcv" "nlme" "nnet" "parallel"
#> [21] "rpart" "spatial" "splines" "stats" "stats4"
#> [26] "survival" "tcltk" "tools" "utils"
packageVersion("Matrix")
#> [1] '1.5.4.1'
# Install TMB from source, now matching the Matrix version
install.packages(c("Rcpp", "RcppEigen"))
#> Installing packages into
'/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpSL7Etm/reprex-bc8b35150024-vague-bull/templib'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#>
/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T//RtmpOpHc0P/downloaded_packages
install.packages("TMB", type = "source", repos =
"https://cloud.r-project.org/";)
#> Installing package into
'/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpSL7Etm/reprex-bc8b35150024-vague-bull/templib'
#> (as 'lib' is unspecified)
# Perform implicit consistency check
packageVersion("Matrix")
#> [1] '1.5.4.1'
requireNamespace("TMB")
#> Loading required namespace: TMB
# Example 2: upgrading Matrix after installing TMB
dir.create("templib")
.libPaths(normalizePath("templib"))
Sys.setenv(R_LIBS_USER = .libPaths()[[1]])
# Sanity check
rownames(installed.packages())
#> [1] "base" "boot" "class" "cluster" "codetools"
#> [6] "compiler" "datasets" "foreign" "graphics" "grDevices"
#> [11] "grid" "KernSmooth" "lattice" "MASS" "Matrix"
#> [16] "methods" "mgcv" "nlme" "nnet" "parallel"
#> [21] "rpart" "spatial" "splines" "stats" "stats4"
#> [26] "survival" "tcltk" "tools" "utils"
packageVersion("Matrix")
#> [1] '1.5.4.1'
# Install TMB from source, now matching the Matrix version
install.packages(c("Rcpp", "RcppEigen"))
#> Installing packages into
'/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpSL7Etm/reprex-bc8b4fff25a6-next-esok/templib'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#>
/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T//RtmpyN9yHv/downloaded_packages
install.packages("TMB", type = "source", repos =
"https://cloud.r-project.org/";)
#> Installing package into
'/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpSL7Etm/reprex-bc8b4fff25a6-next-esok/templib'
#> (as 'lib' is unspecified)
# Upgrading Matrix
install.packages("Matrix")
#> Installing package into
'/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpSL7Etm/reprex-bc8b4fff25a6-next-esok/templib'
#> (as 'lib' is unspecified)
#>
#> The downloaded binary packages are in
#>
/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T//RtmpyN9yHv/downloaded_packages
# Perform implicit consistency check
packageVersion("Matrix")
#> [1] '1.6.1.1'
requireNamespace("TMB")
#> Loading required namespace: TMB
#> Warning in checkMatrixPackageVersion(): Package version inconsistency
detected.
#> TMB was built with Matrix version 1.5.4.1
#> Current Matrix version is 1.6.1.1
#> Please re-install 'TMB' from source using install.packages('TMB',
type = 'source') or ask CRAN for a binary version of 'TMB' matching
CRAN's 'Matrix' package
On 25.09.23 15:22, Simon Urbanek wrote:
> Kirill,
>
> we have no way of knowing when a packages introduced a breaking change (which
> it really shouldn't), so the maintainers would have to inform as (Matrix
> authors did inform us that 1.6.0 introduces breaking change so that's why we
> did a manual re-bulid on that update). Also checking exact version (like TMB
> seems to do) doesn't work for the same reason and is not a good way to deal
> with the problem - we cannot re-build all packages with each change, because
> of the amount of time that would take (by now it takes more that a day to
> re-build CRAN just for one R target) and more fundamental problem with
> re-builds:
>
> The real problem here is deeper than just triggering rebuilds: the users have
> no way of knowing if a package has been re-built and updated, because the
> version does not change. If let's say TMB is release 1.0 used Matrix 1.6.0
> and then Matrix is updated to 1.6.1 then TMB cannot require it because it did
> not bump its version, i.e. the binary for TMB 1.0 with Matrix 1.6.0 is
> indistinguishable from a binary of TMB 1.0 with Matrix 1.6.1 on CRAN so if
> the user used update.packages() it would not count as a new TMB version.
> Therefore TMB really has to update its version from 1.0 to let's say 1.0-1 if
> it requests a rebuild against changes in Matrix 1.6.1. Clearly, this is not
> ideal, but currently there is other way that would guarantee consistency of
> binaries. We are aware of the problem and we are we are in fact thinking
> about possible solutions (special tags on binary versions) for future version
> of R, but for now the package authors have to be aware of potential breakage
> upstream.
>
> Cheers,
> Simon
>
>
>> On 26/09/2023, at 7:46 AM, Kirill Müller via
>> R-SIG-Mac<r-sig-mac@r-project.org> wrote:
>>
>> Hi
>>
>>
>> The TMB package has Matrix in its LinkingTo dependencies. In a clean package
>> library, I see behavior posted below.
>>
>> Should the TMB package have been rebuilt when a new version of the Matrix
>> package was pushed? More general, should we rebuild all reverse LinkingTo
>> dependencies of a package that gets an update on CRAN? This will mean a lot
>> of package rebuilds with Rcpp, but also a lot less frustration down the
>> road. Thanks!
>>
>>
>> Best regards
>>
>> Kirill
>>
>>
>> dir.create("templib")
>> .libPaths(normalizePath("templib"))
>>
>> # Sanity check
>> rownames(installed.packages())
>> #> [1] "base" "boot" "class" "cluster" "codetools"
>> #> [6] "compiler" "datasets" "foreign" "graphics" "grDevices"
>> #> [11] "grid" "KernSmooth" "lattice" "MASS" "Matrix"
>> #> [16] "methods" "mgcv" "nlme" "nnet" "parallel"
>> #> [21] "rpart" "spatial" "splines" "stats" "stats4"
>> #> [26] "survival" "tcltk" "tools" "utils"
>> packageVersion("Matrix")
>> #> [1] '1.5.4.1'
>>
>> # Install fresh binary packages
>> install.packages(c("Matrix", "TMB"))
>> #> Installing packages into
>> '/private/var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T/RtmpXIjIPy/reprex-87aa25cc1fa4-sugar-cob/templib'
>> #> (as 'lib' is unspecified)
>> #> also installing the dependencies 'Rcpp', 'RcppEigen'
>> #>
>> #> The downloaded binary packages are in
>> #>
>> /var/folders/dj/yhk9rkx97wn_ykqtnmk18xvc0000gn/T//Rtmp4GSL7m/downloaded_packages
>> packageVersion("Matrix")
>> #> [1] '1.6.1.1'
>>
>> # Perform implicit consistency check
>> requireNamespace("TMB")
>> #> Loading required namespace: TMB
>> #> Warning in checkMatrixPackageVersion(): Package version inconsistency
>> detected.
>> #> TMB was built with Matrix version 1.6.0
>> #> Current Matrix version is 1.6.1.1
>> #> Please re-install 'TMB' from source using install.packages('TMB', type =
>> 'source') or ask CRAN for a binary version of 'TMB' matching CRAN's 'Matrix'
>> package
>>
>> _______________________________________________
>> R-SIG-Mac mailing list
>> R-SIG-Mac@r-project.org
>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>
[[alternative HTML version deleted]]
_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
