Re: Matrix update triggering need for four rebuilds

[Nilesh Patra]

On 18 November 2023 12:14:37 am IST, Dirk Eddelbuettel  wrote:
>
>On 17 November 2023 at 23:50, Nilesh Patra wrote:
>| 
>| 
>| On 17 November 2023 11:34:21 pm IST, Dirk Eddelbuettel  
>wrote:
>| >
>| >On 17 November 2023 at 18:43, Andreas Tille wrote:
>| >| Am Fri, Nov 17, 2023 at 10:12:02AM -0600 schrieb Dirk Eddelbuettel:
>| >| > Leaving
>| >| > 
>| >| >r-cran-irlba
>| >| >r-cran-openmx
>| >| > 
>| >| > for you (unless you got to it already).
>| >| 
>| >| To make it pretty clear: I will not simply rebuild these packages before
>| >| we have a promising solution for the future.  If you do not agree with
>| >| this you can either
>| >| 
>| >|* Ask for Bin-NMU which should be sufficient
>| >|* Do a team upload
>| >
>| >"Our users are our preference".  Debian Social Contract.
>| 
>| You don't get to play this card with uncoordinated transitions.
>| 
>| Nobody is stopping you from doing team uploads for these packages fwiw.
>
>I tried to explain that upstream (both the package and CRAN) have no real
>handle on this so it is by _definition_ what you refer to (in a denigrating
>tone) an uncoordinated transition.

The transition that's being talked about is the debian one (not the upstream 
one).

This one to be clear:



It is being brought up from time to time of you noticed.

>I'll will retreat and focus on things I
>can control.

Likewise.

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 17 November 2023 at 23:50, Nilesh Patra wrote:
| 
| 
| On 17 November 2023 11:34:21 pm IST, Dirk Eddelbuettel  
wrote:
| >
| >On 17 November 2023 at 18:43, Andreas Tille wrote:
| >| Am Fri, Nov 17, 2023 at 10:12:02AM -0600 schrieb Dirk Eddelbuettel:
| >| > Leaving
| >| > 
| >| >r-cran-irlba
| >| >r-cran-openmx
| >| > 
| >| > for you (unless you got to it already).
| >| 
| >| To make it pretty clear: I will not simply rebuild these packages before
| >| we have a promising solution for the future.  If you do not agree with
| >| this you can either
| >| 
| >|* Ask for Bin-NMU which should be sufficient
| >|* Do a team upload
| >
| >"Our users are our preference".  Debian Social Contract.
| 
| You don't get to play this card with uncoordinated transitions.
| 
| Nobody is stopping you from doing team uploads for these packages fwiw.

I tried to explain that upstream (both the package and CRAN) have no real
handle on this so it is by _definition_ what you refer to (in a denigrating
tone) an uncoordinated transition.

In which a rebuild is good enough for upstream. And I have done one for my
affected packages. My role ends there. If you guys think you must ambush your
users and leave known broken packages in that state then that is your choice.
At least you are saying so out loud.  I'll will retreat and focus on things I
can control.

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Nilesh Patra]

On 17 November 2023 11:34:21 pm IST, Dirk Eddelbuettel  wrote:
>
>On 17 November 2023 at 18:43, Andreas Tille wrote:
>| Am Fri, Nov 17, 2023 at 10:12:02AM -0600 schrieb Dirk Eddelbuettel:
>| > Leaving
>| > 
>| >r-cran-irlba
>| >r-cran-openmx
>| > 
>| > for you (unless you got to it already).
>| 
>| To make it pretty clear: I will not simply rebuild these packages before
>| we have a promising solution for the future.  If you do not agree with
>| this you can either
>| 
>|* Ask for Bin-NMU which should be sufficient
>|* Do a team upload
>
>"Our users are our preference".  Debian Social Contract.

You don't get to play this card with uncoordinated transitions.

Nobody is stopping you from doing team uploads for these packages fwiw.

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 17 November 2023 at 18:43, Andreas Tille wrote:
| Am Fri, Nov 17, 2023 at 10:12:02AM -0600 schrieb Dirk Eddelbuettel:
| > Leaving
| > 
| >r-cran-irlba
| >r-cran-openmx
| > 
| > for you (unless you got to it already).
| 
| To make it pretty clear: I will not simply rebuild these packages before
| we have a promising solution for the future.  If you do not agree with
| this you can either
| 
|* Ask for Bin-NMU which should be sufficient
|* Do a team upload

"Our users are our preference".  Debian Social Contract.

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Andreas Tille]

Am Fri, Nov 17, 2023 at 10:12:02AM -0600 schrieb Dirk Eddelbuettel:
> Leaving
> 
>r-cran-irlba
>r-cran-openmx
> 
> for you (unless you got to it already).

To make it pretty clear: I will not simply rebuild these packages before
we have a promising solution for the future.  If you do not agree with
this you can either

   * Ask for Bin-NMU which should be sufficient
   * Do a team upload

Kind regards
Andreas.

-- 
http://fam-tille.de

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 15 November 2023 at 05:23, Dirk Eddelbuettel wrote:
| 
| On 15 November 2023 at 07:00, Andreas Tille wrote:
| | Am Tue, Nov 14, 2023 at 04:49:01PM -0600 schrieb Dirk Eddelbuettel:
| | > 
| | > On 14 November 2023 at 16:17, Dirk Eddelbuettel wrote:
| | > | 
| | > | On 14 November 2023 at 11:06, Andreas Tille wrote:
| | > | | Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
| | > | | > Most of these are not in Debian but I think we need binary rebuilds 
of
| | > | | > 
| | > | | >irlbabecause of headers
| | > | | >OpenMx   because of headers, a new upstream 2.21.10 
is out too
| | > | | >TMB  because of headers
| | > | | 
| | > | | Uploaded yesterday since I realised the need.
| | > | 
| | > | Thank you!
| | > 
| | > I misread that, I think. So you updated TMB.  Good.  One done, three to 
go.
| | 
| | Yes, that is what I intended to express.
| |  
| | > irlba is widely used, OpenMx is big and I think MatrixModels also pops up.
| | > If you could do a sweep over those I would appreciate it.
| | 
| | I'll happily update these as soon as possible.  However, I'd like to
| | know what might be some sensible means to catch this quickly in future.
| | For r-cran-tmb the strict version checking in d/rules (which is not the
| | best thing to do according to the release team) was raising some signal.
| | 
| | I'd like to clarify first, whether there will be some better solution
| | via some r-matrix-api before I upload those other packages.
| 
| I relayed information I got from Matrix upstream in private email which then
| also lead to the emails written to r-package-devel I already referenced:
| 
|   https://stat.ethz.ch/pipermail/r-package-devel/2023q4/010051.html
|   https://stat.ethz.ch/pipermail/r-package-devel/2023q4/010054.html
| 
| There is nothing more at this point.
| 
| But we know the remaining Debian packages
| 
|   r-cran-irlba
|   r-cran-openmx
|   r-cran-matrixmodels
| 
| require a rebuild.

r-cran-matrixmodels is actually one of mine, and they just put a new upstream
out so I'll update now.  Leaving

   r-cran-irlba
   r-cran-openmx

for you (unless you got to it already).

Dirk

| 
| Dirk
| 
| 
| -- 
| dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 15 November 2023 at 07:00, Andreas Tille wrote:
| Am Tue, Nov 14, 2023 at 04:49:01PM -0600 schrieb Dirk Eddelbuettel:
| > 
| > On 14 November 2023 at 16:17, Dirk Eddelbuettel wrote:
| > | 
| > | On 14 November 2023 at 11:06, Andreas Tille wrote:
| > | | Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
| > | | > Most of these are not in Debian but I think we need binary rebuilds of
| > | | > 
| > | | >irlba  because of headers
| > | | >OpenMx   because of headers, a new upstream 2.21.10 is 
out too
| > | | >TMB  because of headers
| > | | 
| > | | Uploaded yesterday since I realised the need.
| > | 
| > | Thank you!
| > 
| > I misread that, I think. So you updated TMB.  Good.  One done, three to go.
| 
| Yes, that is what I intended to express.
|  
| > irlba is widely used, OpenMx is big and I think MatrixModels also pops up.
| > If you could do a sweep over those I would appreciate it.
| 
| I'll happily update these as soon as possible.  However, I'd like to
| know what might be some sensible means to catch this quickly in future.
| For r-cran-tmb the strict version checking in d/rules (which is not the
| best thing to do according to the release team) was raising some signal.
| 
| I'd like to clarify first, whether there will be some better solution
| via some r-matrix-api before I upload those other packages.

I relayed information I got from Matrix upstream in private email which then
also lead to the emails written to r-package-devel I already referenced:

  https://stat.ethz.ch/pipermail/r-package-devel/2023q4/010051.html
  https://stat.ethz.ch/pipermail/r-package-devel/2023q4/010054.html

There is nothing more at this point.

But we know the remaining Debian packages

  r-cran-irlba
  r-cran-openmx
  r-cran-matrixmodels

require a rebuild.

Dirk


-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Andreas Tille]

Am Tue, Nov 14, 2023 at 04:49:01PM -0600 schrieb Dirk Eddelbuettel:
> 
> On 14 November 2023 at 16:17, Dirk Eddelbuettel wrote:
> | 
> | On 14 November 2023 at 11:06, Andreas Tille wrote:
> | | Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
> | | > Most of these are not in Debian but I think we need binary rebuilds of
> | | > 
> | | >irlbabecause of headers
> | | >OpenMx   because of headers, a new upstream 2.21.10 is 
> out too
> | | >TMB  because of headers
> | | 
> | | Uploaded yesterday since I realised the need.
> | 
> | Thank you!
> 
> I misread that, I think. So you updated TMB.  Good.  One done, three to go.

Yes, that is what I intended to express.
 
> irlba is widely used, OpenMx is big and I think MatrixModels also pops up.
> If you could do a sweep over those I would appreciate it.

I'll happily update these as soon as possible.  However, I'd like to
know what might be some sensible means to catch this quickly in future.
For r-cran-tmb the strict version checking in d/rules (which is not the
best thing to do according to the release team) was raising some signal.

I'd like to clarify first, whether there will be some better solution
via some r-matrix-api before I upload those other packages.

Kind regards
Andreas.

-- 
http://fam-tille.de

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 14 November 2023 at 16:17, Dirk Eddelbuettel wrote:
| 
| On 14 November 2023 at 11:06, Andreas Tille wrote:
| | Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
| | > Most of these are not in Debian but I think we need binary rebuilds of
| | > 
| | >irlba  because of headers
| | >OpenMx   because of headers, a new upstream 2.21.10 is out 
too
| | >TMB  because of headers
| | 
| | Uploaded yesterday since I realised the need.
| 
| Thank you!

I misread that, I think. So you updated TMB.  Good.  One done, three to go.

irlba is widely used, OpenMx is big and I think MatrixModels also pops up.
If you could do a sweep over those I would appreciate it.

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Dirk Eddelbuettel]

On 14 November 2023 at 11:06, Andreas Tille wrote:
| Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
| > Most of these are not in Debian but I think we need binary rebuilds of
| > 
| >irlbabecause of headers
| >OpenMx   because of headers, a new upstream 2.21.10 is out 
too
| >TMB  because of headers
| 
| Uploaded yesterday since I realised the need.

Thank you!

The rest is an open issue and not clear. I had a number of emails with both
the active maintainer of Matrix (Mikael) who is considering declaring an API
version at the package level, and one of the maintainers of an affected
package (my colleague who is upstream for SeuratObject which was bitten by
the S4 signature/namespace caching).  I am also in some emails with CRAN and
R Core but no resolution there.

So for now this is "just an unfortunate one-off" and not yet something ready
for a more systematic change.

| >MatrixModels because of S4 caching
| > 
| > I would appreciate it if someone could tickle rebuilds. To me a quick
| > informal touch of debian/changelog would do; if someone thinks this needs a
| > formal transition go for it.
| 
| In principle upgrading four packages at request is cheap.  However, I
| have the feeling that we are technically more safe if we would introduce
| some r-tmb-api which would technically raise a signal for tmb
| dependencies.  I've "hacked" some workaround into r-cran-tmb since I
| was motivated by the github issue discussing the relation to Matrix[1]
| to fix a very specifix Matrix version.  I've put the release team in
| CC since they were involved into the according discussion.

I did not follow this closely but was that rather 'change in TMB driving
issues in glmmTMB and alike' whereas this email is about 'TMB as a client of
Matrix' ?

Dirk

-- 
dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org

Re: Matrix update triggering need for four rebuilds

[Andreas Tille]

Hi Dirk,

Am Mon, Nov 13, 2023 at 07:23:06AM -0600 schrieb Dirk Eddelbuettel:
> Most of these are not in Debian but I think we need binary rebuilds of
> 
>irlba  because of headers
>OpenMx   because of headers, a new upstream 2.21.10 is out too
>TMB  because of headers

Uploaded yesterday since I realised the need.

>MatrixModels because of S4 caching
> 
> I would appreciate it if someone could tickle rebuilds. To me a quick
> informal touch of debian/changelog would do; if someone thinks this needs a
> formal transition go for it.

In principle upgrading four packages at request is cheap.  However, I
have the feeling that we are technically more safe if we would introduce
some r-tmb-api which would technically raise a signal for tmb
dependencies.  I've "hacked" some workaround into r-cran-tmb since I
was motivated by the github issue discussing the relation to Matrix[1]
to fix a very specifix Matrix version.  I've put the release team in
CC since they were involved into the according discussion.

I consider it necessary to make those dependencies technically
transparent and some according API seems to be the usual way to do this.

> The R Core team and the CRAN maintainers are aware of the implicit problem
> with signalling the need for binary rebuilds. They are discussing this, but
> do not have an answer. Historically, CRAN has informally rebuilt its binaries
> for windows and macOS, but that of course does not help binary distributors
> such as us, other Linux distros, Conda, r2u, ... at all.

May be its interesting to hear the opinion of the said CRAN maintainers
whether they consider the suggested means as appropriate to deal with 
this issue.

Kind regards
Andreas.

[1] https://github.com/kaskr/adcomp/issues/387 

-- 
http://fam-tille.de