Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Sebastian, On Tue, Jan 09, 2024 at 10:49:23AM +0100, Sebastian Ramacher wrote: > > > > > > 22 libobs-dev > > > > > That's a change to the plugin ABI only. > > > > Can you explain how you've reached that conclusion? This is a package > > > > that failed to be analyzed in the latest run; an earlier run against > > > > Ubuntu lunar showed no change in ABI at all. > > > libobs-dev and the shared library are an oddity of obs-studio. There > > > only purpose is to build plugins. > > Ok, but it appears there are a large number of reverse-dependencies on > > libobs0 from other source packages, and there is no OTHER encoding of > > information about plugin ABI aside from this (no Provides: field on > > obs-studio). So what do you suggest here with respect to ABI skew between > > obs-studio and its plugins on armhf? > I need some more time to check the current situation. Have you had enough time to check this out? Are we ok to proceed with handling libobs0 along with the other libraries, which would address the ABI skew despite it technically not being libobs0 whose ABI is changing? > > > > > > 9 libopenmpt-dev > > > > > > > Seems to be a false positive. > > > > > > > > > > > > Your responses here are to the contents of the `sorted-revdep-count` > > > > list. > > > > This is the list of header packages that *we were unable to analyze with > > > > abi-compliance-checker*, and so in the interest of avoiding ABI > > > > mismatches > > > > and broken systems for users when upgrading on 32-bit archs, would get a > > > > package rename to be safe. > > > > This is the plan I had outlined at: > > > > https://lists.debian.org/debian-devel/2023/07/msg00232.html > > > > I am happy for any help in the form of patches to > > > > https://salsa.debian.org/adrien-n/armhf-time_t that fix the failures of > > > > these header packages to be analyzed, or explanations for how a given > > > > header > > > > package's API changes cannot affect the ABI of the runtime libraries > > > > from > > > > the source package so that we can manually exclude those libraries from > > > > the > > > > transition. I am not sure I would *recommend* that maintainers spend a > > > > lot > > > > of time on proving one or another individual library does not have an > > > > ABI > > > > breakage, especially in the long tail of libraries with few > > > > reverse-dependencies whose involvement in the transition is unlikely to > > > > substantially slow down Debian development. > I was looking at the repo but it is unclear to me how packages that can > be skipped are handled there. What would a PR look like to exclude > specific packages? The skip handling is in the block starting at https://salsa.debian.org/vorlon/armhf-time_t/-/blob/main/check-armhf-time_t?ref_type=heads#L852 Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi, On 20-01-2024 23:22, Steve Langasek wrote: So I think an algorithm for deciding the uploads to experimental looks like this: - download source from unstable. - apply the packagename conversion to the source. - grab the debdiff. - submit the NMU diff to the BTS. - download the source again from experimental (possibly a no-op). - apply the debdiff to the source. Except with a *lower* version number than you submitted to the BTS or in two steps below, with a higher version than you submitted to the BTS. (I assume everybody reading this realized that, but just in case.) - if it fails to apply (meaning: debian/control has changed), skip. otherwise, build and upload to experimental. - after packages have cleared binary NEW, upload sourceful NMUs to unstable for all these packages with the same debdiff as before. - if there are any packages included in the uploads to unstable that were skipped for experimental because of different sonames there, deal with binary NEW then in unstable. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sun, Jan 07, 2024 at 09:30:37AM +0100, Rene Engelhard wrote:
> Am 07.01.24 um 02:01 schrieb Steve Langasek:
> > If you say you are going to fix eventual breakage (and not ignoring the
> > test results!) and if that means fixing asm on all affected archs, then
> > it's OK :)
> > Well, yes; though I hope we would see some help from e.g. arm porters if
> > there were actually breakage of this nature.
> Hope doesn't help when it got to the actual problem and this doesn't happen.
Relying on me or any *other* non-ARM porter to fix (hypothetical)
ARM-specific asm issues is also aspirational.
But also, I don't know what else you would propose here. We can't
practically hold libreoffice back from the time_t transition with
dpkg-buildflags overrides, it depends on other libraries that are also
affected (e.g. poppler); and 32-bit time_t is already on borrowed time. It
fails not when the *current* date reaches 2038, but when *any dates you want
to work with on the system* reach 2038. There are known instances already
of software failures due to this limitation, and this will only accelerate
with time. I don't think it's reasonable to postpone the transition.
(Also, the intention here was first posted to debian-devel 6 months ago, and
you didn't raise any objections then? So I don't think a hypothetical
concern in libreoffice asm should block things now.)
> > Alternatively, maybe it would
> > be better to stop shipping libreoffice on 32-bit archs, in that case? I'm
> I'd like to avoid this. Getting rid of i386 is fine, noone needs it, armhf
> is a bit different.
> (rpis etc - even though they could run arm64, but..)
I understand wanting to avoid it, but I think that should be the fallback
position if there are intractable porting problems. Losing libreoffice on
armhf is better than carrying 32-bit time_t forward.
As a data point, Ubuntu will only ship arm64 for raspi in 24.04, not armhf.
> > > > > > - the source packages which need an ABI change
> > > > > > ("source-packages"+"lfs-and-depends-time_t" will have
> > > > > > sourceful NMUs to
> > > > > I get that you probably want NMUs for not needing to ping every
> > > > > maintainer,
> > > > > but this is bad.
> > > > > That e.g. would cause me to upload libreoffice 24.2 rc2 to sid
> > > > > immediately
> > > > > when tagged end of next week to not have this caught in the
> > > > > transition. (see
> > > > > also below for the comment about new upstream versions in
> > > > > experimental.)
> > > > What about the suggestion to not push changes to experimental for
> > > > packages
> > > > that already have new versions in experimental, and do the binary
> > > > package
> > > > renames in unstable instead, leaving the package in experimental alone?
> > > If at all - *both*. At the same time.
> > Most of these packages that are staged in experimental are going to be there
> > because of library transitions...
> And ignore those who use experimental as a testbed of major new upstreams?
> Doesn't sound good.
The purpose of uploading to experimental is twofold.
- to clear binary NEW in an orderly fashion, and minimize the window of
possible misbuilds in unstable once dpkg lands there
- to let the usrmerge tooling run against the packages in experimental and
check for any issues there.
Of course, the second of these doesn't apply to libreoffice.
I hear you that you would like libreoffice to be included because of the
second point.
In another subthread I identified that 130 of the binary packages currently
in experimental are runtime libraries requiring name changes (from 64 source
packages). Since these are explicitly runtime library packages staged in
experimental whose binary package names have NOT changed, they do all need
source NMUs (i.e.: we can't simply promote these packages from experimental
to unstable and rebuild with new dpkg without changing the binary package
names).
This is enough packages that it seems fair to expect them to also have the
binary NEW handling done in experimental, not in unstable.
So I am convinced that we should do NMUs of these to experimental as well,
rather than uploading them straight to unstable.
Packages that already have new sonames in experimental should still NOT have
NMUs uploaded to experimental, because we don't want to add package name
annotations for the new not-yet-in-unstable soname.
So I think an algorithm for deciding the uploads to experimental looks like
this:
- download source from unstable.
- apply the packagename conversion to the source.
- grab the debdiff.
- submit the NMU diff to the BTS.
- download the source again from experimental (possibly a no-op).
- apply the debdiff to the source.
- if it fails to apply (meaning: debian/control has changed), skip.
otherwise, build and upload to experimental.
- after packages have cleared binary NEW, upload sourceful NMUs to unstable
for all these packages with the same debdiff as before.
- if there are any packages
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi again,
On Sun, Jan 07, 2024 at 01:09:48PM +0100, Rene Engelhard wrote:
> Am 07.01.24 um 04:38 schrieb Steve Langasek:
> > The ordering here would be:
> > - dpkg will be uploaded to experimental with 64-bit time_t in the default
> >flags
> > - the source packages which need an ABI change
> >("source-packages"+"lfs-and-depends-time_t") and do not already have
> >versions in experimental, will have sourceful NMUs to experimental with
> >the new binary package names in order to clear binary NEW, in
> > coordination
> > - once these packages have all cleared binary NEW, the new dpkg defaults
> >will be uploaded to unstable
> And in the meanwhile in experimental it will be built with the old time_t on
> other architectures.
> Since the new dpkg won't be picked up.
> Not in the experimental builders unstable+experimental chroots which only
> install experimental packages when Build-Depends: need them.
> (For an undefined time given how much the packages later in the chain wait
> in NEW)
> Especially on armhf which is affected. Or will you do the source NMUs on
> armhf/i386? (For some packages where some features are disabled on 32bit
> this is probably not a good idea)
There is no intention here to do the source NMUs on armhf/i386; they will be
done on amd64.
Yes, autobuilds of these packages in experimental would, in the naive
implementation, still build with the old ABI and the new name.
I am not overly concerned about this, experimental is a staging ground for
developers and not something that folks are intended to install from in
production. And there would be very few if any reverse-dependencies of
these packages uploaded to experimental to pick up the new name. The intent
is to work with the ftp team to batch-accept these through binary NEW once
all of the uploads are done, and then promptly proceed with upgrades to
experimental.
Do you have a different suggestion?
Note that even including a versioned build-dependency on dpkg-dev in the
uploads to experimental doesn't really address the possibility of ABI skew
if reverse-dependencies are uploaded during the critical window where dpkg
has not yet been uploaded to unstable, because *those* packages will not
have a versioned build-dependency on dpkg: so will link against the
libraries with the new names but will themselves be using the old ABI.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Colin,
On Tue, Jan 09, 2024 at 01:32:11PM +, Colin Watson wrote:
> On Mon, Jan 08, 2024 at 03:01:11PM -0800, Steve Langasek wrote:
> > On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote:
> > > On 05-01-2024 17:36, Rene Engelhard wrote:
> > > > Also a problem is that experimental also might already contain totally
> > > > unrelated updates like new upstream versions...
> > > I share this worry. Have you thought about how to handle the cases where
> > > you
> > > don't have experimental to upload to? How big is this problem?
> In the current situation, though, not having experimental available
> means that there's no opportunity for dumat to weigh in regarding
> usrmerge interactions, which seems problematic given Helmut's
> preliminary analysis.
I guess this is a reply to the bits of context I retained above rather than
directly a reply to my most immediate preceding message. In a separate
subthread, I laid out what I thought the process should be for handling the
transition of packages in that situation; but you are raising a separate
point.
Which parallels what Helmut had to say:
> Whenever you face files in aliased locations (other than systemd units),
> please go via experimental to let dumat judge your upload. Check the
> bookworm package for files in aliased locations, not the unstable one.
I have also had other out-of-band conversations with Helmut about the
overlapping issues between usrmerge and time_t, so this was generally
speaking on my radar although I didn't address it in my proposed transition
plan.
I think it is unrealistic to ask experimental to be otherwise quiesced of
transitions to ensure that we can stage all of the affected libraries in
experimental, and I also think that making false transitions for packages
that are ALREADY in experimental because of transitions would be
counterproductive. ("False" because if the library package has a different
soname between unstable and experimental, then renaming the runtime library
package in the *experimental* version is unnecessary because there's nothing
"in the wild" depending on that package name but with the old ABI for which
upgrade compatibility is of concern.)
My preferred path forward here would be, as Helmut suggests, to check which
libraries are in the intersection of (packages in experimental for which we
won't stage time_t uploads) and (packages that exist in bookworm and need
transitioning of aliased paths), and ensure that these packages are handled
carefully according to Helmut's own guidance. My optimistic expectation is
that the intersection will be null, or nearly; but in any case I will bring
data.
And my proposal for checking that set, since we're only talking about
runtime library packages, is to check whether any of the contents of these
packages in bookworm match ^/lib - as a runtime library package NOT matching
this pattern, but matching a pattern for one of the other aliased
directories, would be something ... special.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, Jan 08, 2024 at 03:01:11PM -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote: > > On 05-01-2024 17:36, Rene Engelhard wrote: > > > Also a problem is that experimental also might already contain totally > > > unrelated updates like new upstream versions... > > > I share this worry. Have you thought about how to handle the cases where you > > don't have experimental to upload to? How big is this problem? > > > Another worry, how will you provide the required changes to the maintainers > > of the packages? Via BTS? For those working on salsa: MR? Both? Something > > else? Obviously we should not end in the situation that a new upload goes > > back to the old name (or are the ftp-masters so keen on this transition that > > that won't happen? But what about newer versions with the old name already > > in experimental, conform the former worry?). I've seen NMU's being ignored > > by subsequent uploads by the maintainer, even when they fixed RC issues > > which were then reintroduced. > > I would intend to provide diffs via the BTS. This remains the standard > policy for NMUs in Debian per the Developer's Reference, and as far as I > know has worked effectively for all such previous ABI transitions. In the current situation, though, not having experimental available means that there's no opportunity for dumat to weigh in regarding usrmerge interactions, which seems problematic given Helmut's preliminary analysis. -- Colin Watson (he/him) [cjwat...@debian.org]
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, 08 Jan 2024 at 15:01:11 -0800, Steve Langasek wrote:
> If a maintainer ignores the NMU and drops the rename, they'll be introducing
> a new library transition again on 32-bit archs. Won't they also be caught
> again in binary NEW, for those packages that don't have the same runtime
> library package name in experimental?
To have a concrete example of this, I think you are saying:
- NMU of src:foo renames libfoo0 to libfoo0t64
- maintainer ignores NMU and uploads, effectively renaming libfoo0t64
back to libfoo0
- you want the maintainer's upload to get stuck in NEW
I am not a ftp team member, but if I understand NEW correctly, this
will only trigger a new trip through NEW if the ftp team have already
removed libfoo0 from the overrides file ("decrufting"), which is not
done immediately, only after libfoo0 has not been built by src:foo for
a little while.
If libfoo0 exists in testing and/or stable, I'm not sure whether that
prevents the ftp team's processes from removing it from the overrides file.
If it does, then a new, maintainer upload of libfoo0 will certainly not be
considered NEW, and the safety-catch that you are thinking of will not be
effective.
smcv
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve On 2024-01-05 20:26:56 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 09:32:28PM +0100, Sebastian Ramacher wrote: > > On 2024-01-05 11:06:09 -0800, Steve Langasek wrote: > > > On Fri, Jan 05, 2024 at 06:53:50PM +0100, Sebastian Ramacher wrote: > > > > Hi Steve > > > > > > - perl will also get a sourceful upload, to manually handle 'perlapi' > > > > > Provides. > > > > > Can we combine this one with the upcoming perl transition? See #1055955. > > > > Pros and cons of combining: > > > > - it will save another rebuild of perl modules > > > - new perl upstream versions usually break at least some > > > reverse-dependencies in the archive, so this may slow down the time_t > > > transition to testing for other packages by entangling it with sourceful > > > perl changes. > > > > The release team has already suggested not entangling the two. > > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055955#10 > > > That was two months ago and we were expecting some progress. There was > > however none so far. > > Sorry, what were you expecting exactly? I've had no communication from the > Release Team until now about expectations, including on the debian-devel > thread back in July. Sorry for being unclear. My comment was related to the perl transition. Perl 5.38 will start (ACKed yesterday, it was blocked on our side) before this one and we try to complete it before starting time_t. > > As perl 5.38 is already staged in exeperimental, there are only two > > options: time64_t waits until perl 5.38 is done or we entangle them. > > Rene and Paul raise this concern as well. However, there is another option. > > For any packages involved in this transition which currently have new > versions staged in experimental which are not yet ready to go to unstable, > we can simply exclude them from the upload to experimental, and do the > NEW processing for this subset of packages directly in unstable as part of > the second step. Sounds good to me (provided FPT master are happy to fast track those uploads). > > > > > 22 libobs-dev > > > > > That's a change to the plugin ABI only. > > > > Can you explain how you've reached that conclusion? This is a package > > > that failed to be analyzed in the latest run; an earlier run against > > > Ubuntu lunar showed no change in ABI at all. > > > libobs-dev and the shared library are an oddity of obs-studio. There > > only purpose is to build plugins. > > Ok, but it appears there are a large number of reverse-dependencies on > libobs0 from other source packages, and there is no OTHER encoding of > information about plugin ABI aside from this (no Provides: field on > obs-studio). So what do you suggest here with respect to ABI skew between > obs-studio and its plugins on armhf? I need some more time to check the current situation. > > > > > 9 libopenmpt-dev > > > > > Seems to be a false positive. > > > > > > > > Your responses here are to the contents of the `sorted-revdep-count` list. > > > This is the list of header packages that *we were unable to analyze with > > > abi-compliance-checker*, and so in the interest of avoiding ABI mismatches > > > and broken systems for users when upgrading on 32-bit archs, would get a > > > package rename to be safe. > > > > This is the plan I had outlined at: > > > > https://lists.debian.org/debian-devel/2023/07/msg00232.html > > > > I am happy for any help in the form of patches to > > > https://salsa.debian.org/adrien-n/armhf-time_t that fix the failures of > > > these header packages to be analyzed, or explanations for how a given > > > header > > > package's API changes cannot affect the ABI of the runtime libraries from > > > the source package so that we can manually exclude those libraries from > > > the > > > transition. I am not sure I would *recommend* that maintainers spend a > > > lot > > > of time on proving one or another individual library does not have an ABI > > > breakage, especially in the long tail of libraries with few > > > reverse-dependencies whose involvement in the transition is unlikely to > > > substantially slow down Debian development. I was looking at the repo but it is unclear to me how packages that can be skipped are handled there. What would a PR look like to exclude specific packages? > > > > Change to vlc plugin ABI. This does not require a change of the binary > > > > package name. > > > > There are other reverse-dependencies of libvlccore9 in the archive that > > > are > > > not VLC plugins (phonon4qt5-backend-vlc etc). Are these packages simply > > > mis-linked against that library? > > > No, they are not. The part that changes is exclusiv to plugins. I will > > deal with this change by updating the vlc-plugin-abi-$x Provides. > > This further reduces the count of reverse-dependencies needing rebuilt from > 5452+177 to 5450+178. A net gain of 1 package as a result of you handling > this manually instead for the plugin abi, and it's not even >
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote: > Hi Steve, > On 05-01-2024 17:36, Rene Engelhard wrote: > > Also a problem is that experimental also might already contain totally > > unrelated updates like new upstream versions... > I share this worry. Have you thought about how to handle the cases where you > don't have experimental to upload to? How big is this problem? > Another worry, how will you provide the required changes to the maintainers > of the packages? Via BTS? For those working on salsa: MR? Both? Something > else? Obviously we should not end in the situation that a new upload goes > back to the old name (or are the ftp-masters so keen on this transition that > that won't happen? But what about newer versions with the old name already > in experimental, conform the former worry?). I've seen NMU's being ignored > by subsequent uploads by the maintainer, even when they fixed RC issues > which were then reintroduced. I would intend to provide diffs via the BTS. This remains the standard policy for NMUs in Debian per the Developer's Reference, and as far as I know has worked effectively for all such previous ABI transitions. I think it is reasonable to expect maintainers to pay attention to their bug mail as our defined Debian process. I don't think it would be reasonable to expect people working on cross-archive toolchain transitions to cater to individual maintainer preference in this regard. If a maintainer ignores the NMU and drops the rename, they'll be introducing a new library transition again on 32-bit archs. Won't they also be caught again in binary NEW, for those packages that don't have the same runtime library package name in experimental? It seems to me there's ample opportunity to catch such mistakes if they happen. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, Jan 08, 2024 at 04:12:42PM +0100, Julian Andres Klode wrote: > > - once these packages have all cleared binary NEW, the new dpkg defaults > > will be uploaded to unstable > What happened to the plan to workaround this by doing dak database > shenanigans prior to uploading to avoid binary-NEW altogether, that > I chatted about with mhy/#debian-ftp and you? Did that not work out? That wasn't called out here but is part of the implementation details of the plan. The intention is still to go through binary NEW in experimental before copying to unstable, because it will take time to get all the binary uploads done (longer than it will take to get the sourceful uploads to unstable done), so it's better to stage in experimental to minimize the window in unstable when uploads can be broken. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hey Steve,
On Sat, Jan 06, 2024 at 07:38:42PM -0800, Steve Langasek wrote:
> On Sat, Jan 06, 2024 at 09:25:52AM +0100, Rene Engelhard wrote:
> > Hi,
>
> > Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > > default
> > > > > flags
> > > [...]
> > > What about the suggestion to not push changes to experimental for packages
> > > that already have new versions in experimental, and do the binary package
> > > renames in unstable instead, leaving the package in experimental alone?
>
> > How does that play together with the needed dpkg only in experimental?
>
> > You can't build stuff for unstable involving experimental packages (except
> > manually with binary upload, which would block testing migration)
>
> The ordering here would be:
>
> - dpkg will be uploaded to experimental with 64-bit time_t in the default
> flags
>
> - the source packages which need an ABI change
> ("source-packages"+"lfs-and-depends-time_t") and do not already have
> versions in experimental, will have sourceful NMUs to experimental with
> the new binary package names in order to clear binary NEW, in coordination
>
> - once these packages have all cleared binary NEW, the new dpkg defaults
> will be uploaded to unstable
What happened to the plan to workaround this by doing dak database
shenanigans prior to uploading to avoid binary-NEW altogether, that
I chatted about with mhy/#debian-ftp and you? Did that not work out?
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
signature.asc
Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On 2024-01-05, Sebastian Ramacher wrote: >> libpoppler-cpp0v5 >> libpoppler-glib8 >> libpoppler-qt5-1 >> libpoppler-qt6-3 >> libpoppler126 Poppler core (126ish) changes SONAME by release and is in general not supposed to be used by well-behaving applications. the frontentds (cpp,glib,Qt*) is supposedly stable. But I can confirm that all of them uses time-related types in their api's. (time_t specifically) For added fun, the cpp frontend does it like this on all architectures: typedef unsigned int /* time_t */ time_type; in deprecated api, replaced with time_t in non-deprecated methods. That might require extra investigations. (The commit deprecating and replacing api specifically mentions y2k38) /Sune
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 07.01.24 um 04:38 schrieb Steve Langasek:
The ordering here would be:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t") and do not already have
versions in experimental, will have sourceful NMUs to experimental with
the new binary package names in order to clear binary NEW, in coordination
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
And in the meanwhile in experimental it will be built with the old
time_t on other architectures.
Since the new dpkg won't be picked up.
Not in the experimental builders unstable+experimental chroots which
only install experimental packages when Build-Depends: need them.
(For an undefined time given how much the packages later in the chain
wait in NEW)
Especially on armhf which is affected. Or will you do the source NMUs on
armhf/i386? (For some packages where some features are disabled on 32bit
this is probably not a good idea)
Regards,
Rene
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 07.01.24 um 02:01 schrieb Steve Langasek:
If you say you are going to fix eventual breakage (and not ignoring
the test
results!) and if that means fixing asm on all affected archs, then it's OK
:)
Well, yes; though I hope we would see some help from e.g. arm porters if
there were actually breakage of this nature.
Hope doesn't help when it got to the actual problem and this doesn't happen.
Alternatively, maybe it would
be better to stop shipping libreoffice on 32-bit archs, in that case? I'm
I'd like to avoid this. Getting rid of i386 is fine, noone needs it,
armhf is a bit different.
(rpis etc - even though they could run arm64, but..)
not sure how usable libreoffice is these days on such archs.
popcon.debian.org doesn't appear to have the granularity to tell us if there
actually *are* users; and with only 615 armhf popcon reports (vs 215,562 on
amd64) we don't exactly have a statistically significant sample there.
Only recently I got
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1059158. Maybe UI is
not used much, but I can imagine people using it in paperless-ngx or
other stuff.
I don't really want to bastardize those uses.
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every maintainer,
but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
when tagged end of next week to not have this caught in the transition. (see
also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
If at all - *both*. At the same time.
Most of these packages that are staged in experimental are going to be there
because of library transitions...
And ignore those who use experimental as a testbed of major new
upstreams? Doesn't sound good.
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
(which incidentially contains libreoffice)
Ah! These are packages that have been omitted from the analysis because
they've been manually identified as packages that don't have C or
C++-compatible header files related to a shared library, and therefore even
if they do need updated for 64-bit time_t, they are not affected by CPPFLAGS
and are not part of this transition. (I am not 100% sure this is accurate
for ObjC; in any case ObjC headers are impossible to analyze using
abi-compliance-checker so if ObjC libraries are possibly affected, someone
would have to figure out what to do with them.)
I have no idea on how you indentified it In the libreoffice case that is
not true.
libreoffice-dev-(common) *does* contain C/C++ header files.
And most of them *do* correspond to the libuno* shared packages.
Just that they are not split per library because that wouldn't really
make sense since you need any of them anyway.
And I definitely see
e.g. /usr/include/libreoffice/osl/time.h (libuno-sal3)
No idea whether it actually will be broken by the change, but...
I'm not sure precisely why Adrien found it necessary/useful to add
libreoffice-dev to this exclude list,
Probably because he didn't look at the whole but just at the package
but I can confirm that it's
reasonable, as this particular package contains only a single header file
with #define's and no function prototypes; so in effect it has been manually
analyzed and determined to be unaffected.
But libreoffice-dev-common not (on which libreoffice-dev depends) which
contains all the arch-indep headers. Just libreoffice-dev contains one
header diferent between archs.
Regards,
Rene
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 09:25:52AM +0100, Rene Engelhard wrote:
> Hi,
> Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > default
> > > > flags
> > [...]
> > What about the suggestion to not push changes to experimental for packages
> > that already have new versions in experimental, and do the binary package
> > renames in unstable instead, leaving the package in experimental alone?
> How does that play together with the needed dpkg only in experimental?
> You can't build stuff for unstable involving experimental packages (except
> manually with binary upload, which would block testing migration)
The ordering here would be:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t") and do not already have
versions in experimental, will have sourceful NMUs to experimental with
the new binary package names in order to clear binary NEW, in coordination
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
- source packages which need an ABI change but already have versions in
experimental will be uploaded to unstable, with binaries, to clear binary
NEW
- sourceful NMUs of all the libraries will be reuploaded to unstable
(without binaries, so that they can be promoted to testing without
additional uploads).
- perl will also get a sourceful upload, to manually handle 'perlapi'
Provides.
- binNMUs will be scheduled for all of the reverse-dependencies.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 09:07:15AM +0100, Rene Engelhard wrote:
> Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > default
> > > > flags
> > > I think at that point in time one should know what breaks and whatnot.
> > > Archive rebuild?
> > > (Probably in stages)
> > What kind of breakage are you looking to avoid here?
> build failures/test failures.
> > As mentioned in other points in the thread, regressions as a result of
> > this change should be rare and easy to fix. I do not think it's a good
> > use of time / CPU power to do test rebuilds for this instead of just
> > landing the transition.
> Here especially LibreOffices bridge code worries me.
> That one is tied to ABI and calling conventions. I don't see time_t
> mentioned there but "just" in the public libraries (sal), but I am not sure
> what making a type bigger will cause.
> (And since already
> - i386 needs gcc-12 to build since otherwise the test fails
> - armhf (and other archs like ppc64el and s390x) need Java disabled[1] -
> without any help from any porter to prevent this - I *do* want to avoid
> breakage here.
> If you say you are going to fix eventual breakage (and not ignoring the test
> results!) and if that means fixing asm on all affected archs, then it's OK
> :)
Well, yes; though I hope we would see some help from e.g. arm porters if
there were actually breakage of this nature. Alternatively, maybe it would
be better to stop shipping libreoffice on 32-bit archs, in that case? I'm
not sure how usable libreoffice is these days on such archs.
popcon.debian.org doesn't appear to have the granularity to tell us if there
actually *are* users; and with only 615 armhf popcon reports (vs 215,562 on
amd64) we don't exactly have a statistically significant sample there.
> > > > - the source packages which need an ABI change
> > > > ("source-packages"+"lfs-and-depends-time_t" will have sourceful
> > > > NMUs to
> > > I get that you probably want NMUs for not needing to ping every
> > > maintainer,
> > > but this is bad.
> > > That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
> > > when tagged end of next week to not have this caught in the transition.
> > > (see
> > > also below for the comment about new upstream versions in experimental.)
> > What about the suggestion to not push changes to experimental for packages
> > that already have new versions in experimental, and do the binary package
> > renames in unstable instead, leaving the package in experimental alone?
> If at all - *both*. At the same time.
Most of these packages that are staged in experimental are going to be there
because of library transitions... so I think we definitely don't want to be
renaming those binary packages, they should just drop the t64 suffix when
they move to unstable.
> But yes, that could work.
> (In this specific case I an worrying that the transition will take some
> time, and that I am stuck with 7.6.x instead of uploading 24.2 when it is
> released.)
Ok. I don't think there would be any need for you to wait before uploading
24.2. It might have to wait for time_t for testing migration, but I don't think that should really be long.
> > > > experimental with the new binary package names in order to clear
> > > > binary
> > > > NEW, in coordination
> > > And what about skipped ones? When will those be tried?
> > What do you mean here by "skipped ones"?
> https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
> (which incidentially contains libreoffice)
Ah! These are packages that have been omitted from the analysis because
they've been manually identified as packages that don't have C or
C++-compatible header files related to a shared library, and therefore even
if they do need updated for 64-bit time_t, they are not affected by CPPFLAGS
and are not part of this transition. (I am not 100% sure this is accurate
for ObjC; in any case ObjC headers are impossible to analyze using
abi-compliance-checker so if ObjC libraries are possibly affected, someone
would have to figure out what to do with them.)
I'm not sure precisely why Adrien found it necessary/useful to add
libreoffice-dev to this exclude list, but I can confirm that it's
reasonable, as this particular package contains only a single header file
with #define's and no function prototypes; so in effect it has been manually
analyzed and determined to be unaffected.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 10:01:30AM -0700, Sam Hartman wrote: > > "Steve" == Steve Langasek writes: > >> At one level, krb5-multidev only has an rdep of 5, but I suspect > >> the rdep count for libkrb5-dev is somewhat larger:-) I don't know > >> how many packages would be removed from the transition if we > >> decide most of the krb5 libraries do not need to transition. > Steve> If we determined only libkdb5-10 and libgssrpc4 were > Steve> affected, that does significantly reduce the number of > Steve> packages needing to be rebuilt because of krb5, though maybe > Steve> most of those packages also depend on other libraries that > Steve> are transitioning. > I confirm libgssrpc is also affected. > (It would be nice to get rid of libgssrpc entirely and just use the > system tli-rpc). > My bad; I was so focused on time_t I forgot about timeval. > However, I do think that we're limited only to libkdb5-10 and > libgssrpc4. > I would prefer not to change the abi of the other libraries, especially > libkrb5, libk5crypto3 and libgssapi-krb5-2. > What can I do to help facilitate that? I've updated the override list here. Reduces the count of revdeps to be rebuilt from 5450+168 to 5439+168. Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Drawbacks of lack of mandated packaging workflow (Was: Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline)
On Sun, Jan 07, 2024 at 02:23:32AM +0900, Simon Richter wrote: > > Aren't all these problems just inherent in Debian's lack of a mandated > > packaging tooling and workflow [1,2]? > > We have a mandated tooling and workflow. > > The tooling follows an interface that is defined in Policy. The interface is > deliberately designed to be as flexible as possible. Most packages do not > require this flexibility, which is why a majority use a library of helper > functions that trades that flexibility for ease of use. > > This works because it is a solution that solves 95% of cases, and does not > impose requirements on the remaining 5%. If you wanted 100% of packages to > use this, and turn this into the new interface, then all these corner cases > would need to be handled as well, and the interface extended. > > We also have a version control system -- the Debian archive. It, too, has a > different focus than other version control systems, because it also includes > a mirroring strategy. > > Switching to git would, again, require replication of the missing > functionality, and it would require a lot of work to properly define all > these interfaces and make sure they are extensible in the future. This is, unironically, why we can't have nice things.
Re: Drawbacks of lack of mandated packaging workflow (Was: Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline)
Hi, On 06.01.24 22:15, Gioele Barabucci wrote: Aren't all these problems just inherent in Debian's lack of a mandated packaging tooling and workflow [1,2]? We have a mandated tooling and workflow. The tooling follows an interface that is defined in Policy. The interface is deliberately designed to be as flexible as possible. Most packages do not require this flexibility, which is why a majority use a library of helper functions that trades that flexibility for ease of use. This works because it is a solution that solves 95% of cases, and does not impose requirements on the remaining 5%. If you wanted 100% of packages to use this, and turn this into the new interface, then all these corner cases would need to be handled as well, and the interface extended. We also have a version control system -- the Debian archive. It, too, has a different focus than other version control systems, because it also includes a mirroring strategy. Switching to git would, again, require replication of the missing functionality, and it would require a lot of work to properly define all these interfaces and make sure they are extensible in the future. Simon
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
> "Steve" == Steve Langasek writes: >> At one level, krb5-multidev only has an rdep of 5, but I suspect >> the rdep count for libkrb5-dev is somewhat larger:-) I don't know >> how many packages would be removed from the transition if we >> decide most of the krb5 libraries do not need to transition. Steve> If we determined only libkdb5-10 and libgssrpc4 were Steve> affected, that does significantly reduce the number of Steve> packages needing to be rebuilt because of krb5, though maybe Steve> most of those packages also depend on other libraries that Steve> are transitioning. I confirm libgssrpc is also affected. (It would be nice to get rid of libgssrpc entirely and just use the system tli-rpc). My bad; I was so focused on time_t I forgot about timeval. However, I do think that we're limited only to libkdb5-10 and libgssrpc4. I would prefer not to change the abi of the other libraries, especially libkrb5, libk5crypto3 and libgssapi-krb5-2. What can I do to help facilitate that? --Sam signature.asc Description: PGP signature
Re: Drawbacks of lack of mandated packaging workflow (Was: Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline)
Oops, should have waited sending... On 06-01-2024 14:30, Paul Gevers wrote: On 06-01-2024 14:15, Gioele Barabucci wrote: Aren't all these problems just inherent in Debian's lack of a mandated packaging tooling and workflow [1,2]? Might be, but that doesn't mean that problem goes away. I was talking about bugs, so, only minor. We expect maintainers to track bugs filed in the bts, so that would serve the purpose. But because *most* (according to trends.d.n) are hosted on salsa, an MR might help for an awfully big group. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Drawbacks of lack of mandated packaging workflow (Was: Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline)
Hi Gioele, On 06-01-2024 14:15, Gioele Barabucci wrote: Aren't all these problems just inherent in Debian's lack of a mandated packaging tooling and workflow [1,2]? Might be, but that doesn't mean that problem goes away. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Drawbacks of lack of mandated packaging workflow (Was: Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline)
On 05/01/24 21:17, Paul Gevers wrote: Another worry, how will you provide the required changes to the maintainers of the packages? Via BTS? For those working on salsa: MR? Both? Something else? Obviously we should not end in the situation that a new upload goes back to the old name (or are the ftp-masters so keen on this transition that that won't happen? But what about newer versions with the old name already in experimental, conform the former worry?). I've seen NMU's being ignored by subsequent uploads by the maintainer, even when they fixed RC issues which were then reintroduced. Aren't all these problems just inherent in Debian's lack of a mandated packaging tooling and workflow [1,2]? And the fact that maintainers can decide to maintain their packages in idiosyncratic ways (e.g., no VCS) "just because"? [1] https://salsa.debian.org/debian/grow-your-ideas/-/issues/24 [2] https://salsa.debian.org/debian/grow-your-ideas/-/issues/34 Regards, -- Gioele Barabucci
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi, Am 06.01.24 um 06:51 schrieb Steve Langasek: - dpkg will be uploaded to experimental with 64-bit time_t in the default flags [...] What about the suggestion to not push changes to experimental for packages that already have new versions in experimental, and do the binary package renames in unstable instead, leaving the package in experimental alone? How does that play together with the needed dpkg only in experimental? You can't build stuff for unstable involving experimental packages (except manually with binary upload, which would block testing migration) Regards, Rene
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 12:23:00AM -0800, Steve Langasek wrote: > I am also attaching here the dd-list output for the packages that will need > to be sourcefully NMUed for the transition, for your review. I could readily identify a number of packages (incomplete) also affected by DEP17. Whenever you face files in aliased locations (other than systemd units), please go via experimental to let dumat judge your upload. Check the bookworm package for files in aliased locations, not the unstable one. >uhd DEP17-affected, probably harmless >btrfs-progs DEP17-affected >pacemaker (U) DEP17-affected >libmtp DEP17-affected, probably harmless >openafs (U) DEP17-affected, probably harmless >samba (U) DEP17-affected, probably harmless >apt DEP17-affected, probably harmless >ceph DEP17-affected >util-linux (U) Do not upload to unstable directly. Will need mitigations. >libapogee3 Do not upload to unstable directly. Will need mitigations. >libfishcamp Do not upload to unstable directly. Will need mitigations. >libplayerone Do not upload to unstable directly. Will need mitigations. >libricohcamerasdk Do not upload to unstable directly. Will need mitigations. >libsbig Do not upload to unstable directly. Will need mitigations. >boinc DEP17-affected >gcc-13 If this affects libgcc-s1, do not upload to unstable as you risk deleting libgcc_s.so.1. Will need mitigations in that case. >libosmo-sccp DEP17-affected, probably harmless >libgpod DEP17-affected, probably harmless >libselinux Do not upload to unstable directly. Will need mitigations. >zfs-linux Do not upload to unstable directly. Will need mitigations. >libguestfs (U) DEP17-affected, probably harmless >libtirpc Do not upload to unstable directly. Will need mitigations. >fuse >fuse3 Do not upload to unstable directly. Will need mitigations. >audit Do not upload to unstable directly. Will need mitigations. >zlib Do not upload to unstable. Will cause file loss unless mitigated. >readline Do not upload to unstable. Will cause file loss unless mitigated. >openrc (U) Do not upload to unstable directly. Will need mitigations. >krb5 DEP17-affected As predicted, this is going to have annoying interactions and the list here definitely is incomplete. Helmut
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve,
Am 06.01.24 um 06:51 schrieb Steve Langasek:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
I think at that point in time one should know what breaks and whatnot.
Archive rebuild?
(Probably in stages)
What kind of breakage are you looking to avoid here? As mentioned in other
build failures/test failures.
points in the thread, regressions as a result of this change should be rare
and easy to fix. I do not think it's a good use of time / CPU power to do
test rebuilds for this instead of just landing the transition.
Here especially LibreOffices bridge code worries me.
That one is tied to ABI and calling conventions. I don't see time_t
mentioned there but "just" in the public libraries (sal), but I am not
sure what making a type bigger will cause.
(And since already
- i386 needs gcc-12 to build since otherwise the test fails
- armhf (and other archs like ppc64el and s390x) need Java disabled[1]
- without any help from any porter to prevent this - I *do* want to
avoid breakage here.
If you say you are going to fix eventual breakage (and not ignoring the
test results!) and if that means fixing asm on all affected archs, then
it's OK :)
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every maintainer,
but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
when tagged end of next week to not have this caught in the transition. (see
also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
If at all - *both*. At the same time.
But yes, that could work.
(In this specific case I an worrying that the transition will take some
time, and that I am stuck with 7.6.x instead of uploading 24.2 when it
is released.)
libreoffice libs only have one reverse-dep, zemberek-ooo; so the risk of
entanglement here is small anyway.
Except in poppler etc. transitions.. But yeah, nothing really uses the
C++ libs nowadays.
I think the above proposal, to skip packages already in experimental from
the set of uploads to experimental, would address your concern. It's not as
if there is going to be any time that it's ok to tell maintainers they can't
use experimental at all because we're doing this transition.
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
(which incidentially contains libreoffice)
Regards,
Rene
[1] Ubuntu just ignores the test failures. No, not an option.
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 02:23:59PM -0700, Sam Hartman wrote: > > "Steve" == Steve Langasek writes: > Steve> - In multi-library packages, there is no reliable way to map > Steve> from a set of headers in a dev package to specific shared > Steve> libraries in a runtime library package that's not > Steve> additionally computationally prohibitive; we therefore > Steve> conservatively assume that if any headers from a source > Steve> package show time_t ABI changes, all the runtime library > Steve> packages from the source package are affected by the > Steve> transition. This seems sensible in general, but er, in the > Steve> pathological case we have Source: zlib that now ships both > Steve> zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not > Steve> be ABI breaking, libminizip-dev has failed to analyze, and we > Steve> don't want to force a transition for zlib1g because of > Steve> libminizip (!). Current plan is to simply special case > Steve> zlib1g, but there could be other problem packages we haven't > Steve> identified. > I think krb5 may be in this category. > In particular, it looks like time_t is only exposed in the kdb.h API, > which appears to have no reverse depends outside of krb5. > I actually think that the freeIPA server may have a plugin that depends > on kdb.h, and Samba4 can be built in such a way that it depends on > kdb.h, but it looks like neither of those applies to the Debian archive. https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/compat_reports/libkrb5-dev/lfs_to_time_t/compat_report.html indicates that the CLIENT struct is affected because the cl_call function pointer takes a 'struct timeval' as an argument. So that's not internal to kdb, it looks like an actual ABI change? > At one level, krb5-multidev only has an rdep of 5, but I suspect the > rdep count for libkrb5-dev is somewhat larger:-) > I don't know how many packages would be removed from the transition if > we decide most of the krb5 libraries do not need to transition. If we determined only libkdb5-10 and libgssrpc4 were affected, that does significantly reduce the number of packages needing to be rebuilt because of krb5, though maybe most of those packages also depend on other libraries that are transitioning. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 05:28:37PM +0100, Guillem Jover wrote: > > Guillem Jover > >libaio > >libmd > >liburing > I checked these, and it looks like libmd and liburing are > false-positives? > * libmd uses AC_SYS_LARGEFILE, and on 32-bit arches it is already built >with LFS, the problem is that the header exposes off_t which means >the code linking against it needs to match its build flags, >otherwise they would already be broken now. You might want to look >into this as a potential pattern for other false-positives probably. >(I should probably update upstream the .pc file to include the >-D_FILE_OFFSET_BITS=64 flags if enabled, but I don't think the >analysis used .pc files anyway.) Thanks - yes, the analysis didn't use .pc files, though I'm checking if that's something we could improve on. But as you say it doesn't help with the current libmd anyway, so I'm adding that to the manual exclusion list. Reduces the count of revdeps to be rebuilt from 5450+178 to 5450+169. > * liburing is marked as LFS-sensitive, but that comes from inline >functions using off_t which end up casting that into an u32 type, >so I don't think this should affect the ABI. It is also forcibly >built with -D_FILE_OFFSET_BITS=64 in the upstream build system >(and with future=+lfs for good measure in the packaging, which >I'll switch to abi=+lfs). io_uring_prep_fadvise and io_uring_prep_madvise appear to be symbols publicly exposed in the ABI of /usr/lib/x86_64-linux-gnu/liburing-ffi.so.2 so I don't think it's true that the marking is only because of inline functions? But as this is also already built with LFS and is a false-positive, I've removed this as well. Reduces the count of revdeps to be rebuilt from 5450+169 to 5450+168 (not much improvement :). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 05:36:10PM +0100, Rene Engelhard wrote:
> > My strawman proposal is to give this thread 2 weeks from today for feedback
> > and further refinement, and also to further reduce the number of
> > false-positives included in the transition. Then, starting on Jan 18:
> > - dpkg will be uploaded to experimental with 64-bit time_t in the default
> >flags
> I think at that point in time one should know what breaks and whatnot.
> Archive rebuild?
> (Probably in stages)
What kind of breakage are you looking to avoid here? As mentioned in other
points in the thread, regressions as a result of this change should be rare
and easy to fix. I do not think it's a good use of time / CPU power to do
test rebuilds for this instead of just landing the transition.
> > - the source packages which need an ABI change
> >("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
> I get that you probably want NMUs for not needing to ping every maintainer,
> but this is bad.
> That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
> when tagged end of next week to not have this caught in the transition. (see
> also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
libreoffice libs only have one reverse-dep, zemberek-ooo; so the risk of
entanglement here is small anyway.
I think the above proposal, to skip packages already in experimental from
the set of uploads to experimental, would address your concern. It's not as
if there is going to be any time that it's ok to tell maintainers they can't
use experimental at all because we're doing this transition.
> >experimental with the new binary package names in order to clear binary
> >NEW, in coordination
> And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 09:32:28PM +0100, Sebastian Ramacher wrote: > On 2024-01-05 11:06:09 -0800, Steve Langasek wrote: > > On Fri, Jan 05, 2024 at 06:53:50PM +0100, Sebastian Ramacher wrote: > > > Hi Steve > > > > - perl will also get a sourceful upload, to manually handle 'perlapi' > > > > Provides. > > > Can we combine this one with the upcoming perl transition? See #1055955. > > Pros and cons of combining: > > - it will save another rebuild of perl modules > > - new perl upstream versions usually break at least some > > reverse-dependencies in the archive, so this may slow down the time_t > > transition to testing for other packages by entangling it with sourceful > > perl changes. > > The release team has already suggested not entangling the two. > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055955#10 > That was two months ago and we were expecting some progress. There was > however none so far. Sorry, what were you expecting exactly? I've had no communication from the Release Team until now about expectations, including on the debian-devel thread back in July. > As perl 5.38 is already staged in exeperimental, there are only two > options: time64_t waits until perl 5.38 is done or we entangle them. Rene and Paul raise this concern as well. However, there is another option. For any packages involved in this transition which currently have new versions staged in experimental which are not yet ready to go to unstable, we can simply exclude them from the upload to experimental, and do the NEW processing for this subset of packages directly in unstable as part of the second step. This should be an ABI change but not an API change; the rate of new build failures should be very small (small enough that I have not proposed any rebuild testing as part of this transition). As previously discussed, there may be some impact from -Werror=implicit-function-declaration but those can be quickly dealt with. Barring any existing testing migration blockages, I expect this whole assembly to be able to migrate to testing rather quickly. The main point is that we don't want to have a large window between landing dpkg in unstable, and uploading the libraries in unstable, due to NEW processing; because that will result in misbuilt and crashy combinations of packages on armhf until resolved. And if the time_t transition goes into unstable, has not yet migrated to testing, and some of the library transitions in question are ready to go to unstable, I don't mind; I'm happy to work with the maintainers of those libraries to get the transitions done, and also they can just ignore the previous NMUs because they're getting a new SONAME anyway. Does this seem practicable to you (and the release team generally)? > > > > 22 libobs-dev > > > That's a change to the plugin ABI only. > > Can you explain how you've reached that conclusion? This is a package > > that failed to be analyzed in the latest run; an earlier run against > > Ubuntu lunar showed no change in ABI at all. > libobs-dev and the shared library are an oddity of obs-studio. There > only purpose is to build plugins. Ok, but it appears there are a large number of reverse-dependencies on libobs0 from other source packages, and there is no OTHER encoding of information about plugin ABI aside from this (no Provides: field on obs-studio). So what do you suggest here with respect to ABI skew between obs-studio and its plugins on armhf? > > Looks like the failure to analyze should be easily fixable (maybe libobs-dev > > shouldn't be shipping this header?) > > https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/logs/libobs-dev/base/log.txt > > > > 14 gnuradio > > > gnuradio bump its SONAME every other month. > > And usually takes time, at least from what I've seen on the Ubuntu side, > > to settle all reverse-dependencies out into a consistent state where > > they can migrate to testing > I wouldn't waste my time with gnuradio. The maintainers does not > coordinate transitions. After January 10th assuming that AUTORM kicks > in, it won't be relevant any more. It wastes less of my time to *not* special-case it, then ;-) > > > > 9 libopenmpt-dev > > > Seems to be a false positive. > > > > Your responses here are to the contents of the `sorted-revdep-count` list. > > This is the list of header packages that *we were unable to analyze with > > abi-compliance-checker*, and so in the interest of avoiding ABI mismatches > > and broken systems for users when upgrading on 32-bit archs, would get a > > package rename to be safe. > > This is the plan I had outlined at: > > https://lists.debian.org/debian-devel/2023/07/msg00232.html > > I am happy for any help in the form of patches to > > https://salsa.debian.org/adrien-n/armhf-time_t that fix the failures of > > these header packages to be analyzed, or explanations for how a given header > > package's API changes cannot affect the ABI of the runtime libraries from > > the source
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 06:59:20PM +0100, Sebastian Ramacher wrote: > > > > I am also attaching here the dd-list output for the packages that will > > > > need > > > > to be sourcefully NMUed for the transition, for your review. > > > Why do the need sourceful NMUs if they just need to be rebuilt? > > Sorry, if the original message hadn't been lost somewhere in the mail > > system before being delivered to debian-devel (I've now tried to resend it), > > this might have been clearer from context. Guillem points out the mail has > > been delivered to debian-release, so you can read the whole thing there: > > https://lists.debian.org/debian-release/2024/01/msg00033.html > > Anyway, this is the list of source packages containing libraries whose ABI > > will change. So the packages need to be renamed in order to expose the ABI > > incompatibility to reverse-dependencies. > I am confused. Above you say: > > these in turn have 174 additional > > reverse-dependencies that would need rebuilt (list attached). > This sounds to me like those are packages that are involved in the > transition and need rebuilds, but do not change their ABI. And in fact, > for most of packages that I maintain on the list, the ABI does not > change. > Can you please clarify which of the packages in your lists require > changes to the binary package names and which do not? Sorry for the confusion. The two lists requiring binary package name changes are the attachments named 'source-packages' and 'lfs-and-depends-time_t'. This is what I fed into dd-list, and encompass 1248 source packages (1195 + 53). -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
> "Steve" == Steve Langasek writes: Steve> - In multi-library packages, there is no reliable way to map Steve> from a set of headers in a dev package to specific shared Steve> libraries in a runtime library package that's not Steve> additionally computationally prohibitive; we therefore Steve> conservatively assume that if any headers from a source Steve> package show time_t ABI changes, all the runtime library Steve> packages from the source package are affected by the Steve> transition. This seems sensible in general, but er, in the Steve> pathological case we have Source: zlib that now ships both Steve> zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not Steve> be ABI breaking, libminizip-dev has failed to analyze, and we Steve> don't want to force a transition for zlib1g because of Steve> libminizip (!). Current plan is to simply special case Steve> zlib1g, but there could be other problem packages we haven't Steve> identified. I think krb5 may be in this category. In particular, it looks like time_t is only exposed in the kdb.h API, which appears to have no reverse depends outside of krb5. I actually think that the freeIPA server may have a plugin that depends on kdb.h, and Samba4 can be built in such a way that it depends on kdb.h, but it looks like neither of those applies to the Debian archive. At one level, krb5-multidev only has an rdep of 5, but I suspect the rdep count for libkrb5-dev is somewhat larger:-) I don't know how many packages would be removed from the transition if we decide most of the krb5 libraries do not need to transition. signature.asc Description: PGP signature
64-bit time_t: updated archive analysis, proposed transition plan with timeline
[Sorry for breaking the thread; my previous message exceeded a size limit for debian-devel so I'm having to re-send with some of the attachments removed and put on a website instead. This mail also includes an updated 'sorted-revdep-count' list; I inadvertently had a stale local mirror in my sid apt config that was causing these numbers to be inflated, the new attachment is better.] Hi folks, Once again, coming back around to the question of an archive-wide migration for time_t[0]. We have a clear path forward on the toolchain side (https://bugs.debian.org/1037136), and we have an updated ABI analysis done against current Debian unstable as of 2023-12-18, superseding the previous preliminary analysis we had done on Ubuntu mantic. This re-analysis took a while to get done in large part because the earlier mailing list thread revealed gaps in the identification of "dev" packages[1], so we spent some time fixing this to ensure we had proper archive coverage. Output files from the new analysis can be found here: https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/ == Description of methodology == Before going into details of the plan I'm proposing for the migration, I want to surface to the list the methodology used for the analysis, so that people have an opportunity to identify any errors. - Packages are identified as "dev" packages if, according to Contents-armhf in the archive, they ship files with an extension of .h, .hpp, .hxx, or .hh in any directory outside of /usr/share/doc. - Packages are identified as "library" packages if, according to Contents, they ship files with an extension of .so or match the pattern *.so.*. - Source packages are candidates for ABI analysis as part of this transition if at least one of their binary packages is a "dev" package, at least one is a "library" package, and at least one ships a shlibs control file. - This means packages that provide plugin APIs are excluded from the transition (they couldn't be automated anyway), but might still have ABI breakage that will require handling. (ex: apache2-dev; but it looks like perl did get picked up correctly by way of libperl5.36) - In multi-library packages, there is no reliable way to map from a set of headers in a dev package to specific shared libraries in a runtime library package that's not additionally computationally prohibitive; we therefore conservatively assume that if any headers from a source package show time_t ABI changes, all the runtime library packages from the source package are affected by the transition. This seems sensible in general, but er, in the pathological case we have Source: zlib that now ships both zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not be ABI breaking, libminizip-dev has failed to analyze, and we don't want to force a transition for zlib1g because of libminizip (!). Current plan is to simply special case zlib1g, but there could be other problem packages we haven't identified. - Packages that could not be analyzed for whatever reason are still assumed to have an ABI that's sensitive to time_t and have to be included in the transition. Happily, due to improvements in this run of the number of packages that could successfully be analyzed, the number of libraries in this category has dropped from 1541 to 929, of which 69 have no reverse-dependencies at all. The final list of these header packages that could not be analyzed is attached, in case anyone still wants to identify that a library on this list whose ABI is not affected by time_t and should therefore be excluded from the transition. Note, however, that no effort has been made to filter out dev packages from this list that come from source packages containing OTHER dev packages that are known to have ABI breakage; for a number of the packages listed, further analysis would not change the outcome. (e.g. qt5-qmake failed to be analyzed, but qtbase-opensource-src also ships qtbase5-private-dev which shows time_t ABI sensitivity.) == Results == The overall findings of this analysis are 1,745 "dev" packages which either are confirmed to have ABI changes or could not be checked; mapping to 2,154 runtime libraries[2] from 1,195 source packages (list attached) and 5,477 reverse-dependencies requiring no-change rebuilds[3]. This is within the previously calculated range of "5300 to 5600", but there are a number of newly-identified packages that fail to compile and have a large number of reverse-dependencies. I will continue to work to identify false-positives here in the hopes of bringing this count down before pulling the trigger on an actual transition. My understanding is that this will also result in a perl ABI transition, separately from the libperl ABI transition; since this dependency isn't calculated via shlibs it was not automatically included in this analysis. This seems to be at most about 600 additional packages to be included in
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On 2024-01-05 11:06:09 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 06:53:50PM +0100, Sebastian Ramacher wrote: > > Hi Steve > > > > - perl will also get a sourceful upload, to manually handle 'perlapi' > > > Provides. > > > Can we combine this one with the upcoming perl transition? See #1055955. > > Pros and cons of combining: > > - it will save another rebuild of perl modules > - new perl upstream versions usually break at least some > reverse-dependencies in the archive, so this may slow down the time_t > transition to testing for other packages by entangling it with sourceful > perl changes. > > The release team has already suggested not entangling the two. > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055955#10 That was two months ago and we were expecting some progress. There was however none so far. As perl 5.38 is already staged in exeperimental, there are only two options: time64_t waits until perl 5.38 is done or we entangle them. The same is true for all the other transitions (poppler, g2clib, ...) that are currently staged in experimental. So unless we want to tell everyone to stop preparing transitions in experimental (which is contrary to what we, the RT, tell maintainers to do all the time), get everything from experimental removed, etc, I expect that we have to entangle time64_t and all transitions that are staged in experimental. > > Here are some more comments on individual packages. > > > > 22 libobs-dev > > > That's a change to the plugin ABI only. > > Can you explain how you've reached that conclusion? This is a package that > failed to be analyzed in the latest run; an earlier run against Ubuntu lunar > showed no change in ABI at all. libobs-dev and the shared library are an oddity of obs-studio. There only purpose is to build plugins. > > Looks like the failure to analyze should be easily fixable (maybe libobs-dev > shouldn't be shipping this header?) > https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/logs/libobs-dev/base/log.txt > > > > 14 gnuradio > > > gnuradio bump its SONAME every other month. > > And usually takes time, at least from what I've seen on the Ubuntu side, to > settle all reverse-dependencies out into a consistent state where they can > migrate to testing I wouldn't waste my time with gnuradio. The maintainers does not coordinate transitions. After January 10th assuming that AUTORM kicks in, it won't be relevant any more. > > > 9 libopenmpt-dev > > > Seems to be a false positive. > > > > Your responses here are to the contents of the `sorted-revdep-count` list. > This is the list of header packages that *we were unable to analyze with > abi-compliance-checker*, and so in the interest of avoiding ABI mismatches > and broken systems for users when upgrading on 32-bit archs, would get a > package rename to be safe. > > This is the plan I had outlined at: > > https://lists.debian.org/debian-devel/2023/07/msg00232.html > > I am happy for any help in the form of patches to > https://salsa.debian.org/adrien-n/armhf-time_t that fix the failures of > these header packages to be analyzed, or explanations for how a given header > package's API changes cannot affect the ABI of the runtime libraries from > the source package so that we can manually exclude those libraries from the > transition. I am not sure I would *recommend* that maintainers spend a lot > of time on proving one or another individual library does not have an ABI > breakage, especially in the long tail of libraries with few > reverse-dependencies whose involvement in the transition is unlikely to > substantially slow down Debian development. Based on the number of false positives in the libraries that I maintain, the time it takes to prepare, upload, accept from binNEW, etc. with involvement from different teams, I wonder if would not be more efficient to give maintainers some time to check their packages. > > > 5 gcc-10-plugin-dev > > > Can be skipped, not part of testing. Should be RMed from the archive > > instead. > > Thanks, I had already manually excluded gcc-13-plugin-dev from the > transition, but there are gcc-*-plugin-dev in the archive for 4 other > versions of gcc. I've updated things here to exclude all of them now. > > I will not, in general, exclude a library from the transition only because > it's currently not in testing; this could be a transient situation, and > users' shouldn't wind up with broken partial upgrades of this library later > if it returns to testing. I added a hint so that gcc-10 cannot migrate back to testing. > > > 4 llvm-15-dev > > > llvm-toolchain-15 is scheduled for removal. Reverse dependencies should > > get an RC bug instead. > > > > libavutil58 > > > av_timegm is not used by any package in the archive. I'd skip ffmpeg and > > it's libraries. > > How did you determine that it's unused by any reverse-dependencies? I'd be > happy to exclude it, but want to be sure we're not exposing users to bugs in >
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve, On 05-01-2024 17:36, Rene Engelhard wrote: Also a problem is that experimental also might already contain totally unrelated updates like new upstream versions... I share this worry. Have you thought about how to handle the cases where you don't have experimental to upload to? How big is this problem? Another worry, how will you provide the required changes to the maintainers of the packages? Via BTS? For those working on salsa: MR? Both? Something else? Obviously we should not end in the situation that a new upload goes back to the old name (or are the ftp-masters so keen on this transition that that won't happen? But what about newer versions with the old name already in experimental, conform the former worry?). I've seen NMU's being ignored by subsequent uploads by the maintainer, even when they fixed RC issues which were then reintroduced. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 06:53:50PM +0100, Sebastian Ramacher wrote: > Hi Steve > > - perl will also get a sourceful upload, to manually handle 'perlapi' > > Provides. > Can we combine this one with the upcoming perl transition? See #1055955. Pros and cons of combining: - it will save another rebuild of perl modules - new perl upstream versions usually break at least some reverse-dependencies in the archive, so this may slow down the time_t transition to testing for other packages by entangling it with sourceful perl changes. The release team has already suggested not entangling the two. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1055955#10 > Here are some more comments on individual packages. > > 22 libobs-dev > That's a change to the plugin ABI only. Can you explain how you've reached that conclusion? This is a package that failed to be analyzed in the latest run; an earlier run against Ubuntu lunar showed no change in ABI at all. Looks like the failure to analyze should be easily fixable (maybe libobs-dev shouldn't be shipping this header?) https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/logs/libobs-dev/base/log.txt > > 14 gnuradio > gnuradio bump its SONAME every other month. And usually takes time, at least from what I've seen on the Ubuntu side, to settle all reverse-dependencies out into a consistent state where they can migrate to testing > > 9 libopenmpt-dev > Seems to be a false positive. Your responses here are to the contents of the `sorted-revdep-count` list. This is the list of header packages that *we were unable to analyze with abi-compliance-checker*, and so in the interest of avoiding ABI mismatches and broken systems for users when upgrading on 32-bit archs, would get a package rename to be safe. This is the plan I had outlined at: https://lists.debian.org/debian-devel/2023/07/msg00232.html I am happy for any help in the form of patches to https://salsa.debian.org/adrien-n/armhf-time_t that fix the failures of these header packages to be analyzed, or explanations for how a given header package's API changes cannot affect the ABI of the runtime libraries from the source package so that we can manually exclude those libraries from the transition. I am not sure I would *recommend* that maintainers spend a lot of time on proving one or another individual library does not have an ABI breakage, especially in the long tail of libraries with few reverse-dependencies whose involvement in the transition is unlikely to substantially slow down Debian development. > > 5 gcc-10-plugin-dev > Can be skipped, not part of testing. Should be RMed from the archive > instead. Thanks, I had already manually excluded gcc-13-plugin-dev from the transition, but there are gcc-*-plugin-dev in the archive for 4 other versions of gcc. I've updated things here to exclude all of them now. I will not, in general, exclude a library from the transition only because it's currently not in testing; this could be a transient situation, and users' shouldn't wind up with broken partial upgrades of this library later if it returns to testing. > > 4 llvm-15-dev > llvm-toolchain-15 is scheduled for removal. Reverse dependencies should > get an RC bug instead. > > libavutil58 > av_timegm is not used by any package in the archive. I'd skip ffmpeg and > it's libraries. How did you determine that it's unused by any reverse-dependencies? I'd be happy to exclude it, but want to be sure we're not exposing users to bugs in the process. > > libpoppler-cpp0v5 > > libpoppler-glib8 > > libpoppler-qt5-1 > > libpoppler-qt6-3 > > libpoppler126 > Can be combined with the upcoming poppler transiton (see #1060019) Again, combining the time_t transition with transitions introducing sourceful changes (and possible API incompatibilities) to existing libraries risks slowing the transition down to Debian's detriment. > > libvlc5 > > libvlccore9 > Change to vlc plugin ABI. This does not require a change of the binary > package name. There are other reverse-dependencies of libvlccore9 in the archive that are not VLC plugins (phonon4qt5-backend-vlc etc). Are these packages simply mis-linked against that library? Thanks, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve On 2024-01-05 09:42:06 -0800, Steve Langasek wrote: > Hi Sebastian, > > On Fri, Jan 05, 2024 at 06:34:38PM +0100, Sebastian Ramacher wrote: > > On 2024-01-05 00:23:00 -0800, Steve Langasek wrote: > > > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > > > == Results == > > > > > The overall findings of this analysis are 1,745 "dev" packages which > > > > either are confirmed to have ABI changes or could not be checked; > > > > mapping to 2,154 runtime libraries (list attached) from 1,195 source > > > > packages (list attached) and 5,477 reverse-dependencies requiring > > > > no-change rebuilds (list attached). This is within the previously > > > > calculated range of "5300 to 5600", but there are a number of > > > > newly-identified packages that fail to compile and have a large number > > > > of reverse-dependencies. I will continue to work to identify > > > > false-positives here in the hopes of bringing this count down before > > > > pulling the trigger on an actual transition. > > > > [...] > > > > > In addition, Guillem pointed out that if there are libraries whose > > > > ABIs are lfs-sensitive but not time_t-sensitive, and either they > > > > themselves depend on libraries which are time_t-sensitive or they have > > > > reverse-dependencies that do, so they will also need to be included in > > > > the transition. I have identified a list of 53 packages in this > > > > category (list attached); these in turn have 174 additional > > > > reverse-dependencies that would need rebuilt (list attached). > > > > I am also attaching here the dd-list output for the packages that will > > > need > > > to be sourcefully NMUed for the transition, for your review. > > > Why do the need sourceful NMUs if they just need to be rebuilt? > > Sorry, if the original message hadn't been lost somewhere in the mail > system before being delivered to debian-devel (I've now tried to resend it), > this might have been clearer from context. Guillem points out the mail has > been delivered to debian-release, so you can read the whole thing there: > > https://lists.debian.org/debian-release/2024/01/msg00033.html > > Anyway, this is the list of source packages containing libraries whose ABI > will change. So the packages need to be renamed in order to expose the ABI > incompatibility to reverse-dependencies. I am confused. Above you say: > these in turn have 174 additional > reverse-dependencies that would need rebuilt (list attached). This sounds to me like those are packages that are involved in the transition and need rebuilds, but do not change their ABI. And in fact, for most of packages that I maintain on the list, the ABI does not change. Can you please clarify which of the packages in your lists require changes to the binary package names and which do not? Cheers -- Sebastian Ramacher
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve > - perl will also get a sourceful upload, to manually handle 'perlapi' > Provides. Can we combine this one with the upcoming perl transition? See #1055955. Here are some more comments on individual packages. > 22 libobs-dev That's a change to the plugin ABI only. > 14 gnuradio gnuradio bump its SONAME every other month. > 9 libopenmpt-dev Seems to be a false positive. > 9 libogre-1.9-dev > 9 libgirara-dev Why is this one on the list? > 5 gcc-10-plugin-dev Can be skipped, not part of testing. Should be RMed from the archive instead. > 4 llvm-15-dev llvm-toolchain-15 is scheduled for removal. Reverse dependencies should get an RC bug instead. > 4 libspatialaudio-dev Why is this package on the list? > 3 libdvbpsi-dev Seems to be a false positive > libavutil58 av_timegm is not used by any package in the archive. I'd skip ffmpeg and it's libraries. > libpoppler-cpp0v5 > libpoppler-glib8 > libpoppler-qt5-1 > libpoppler-qt6-3 > libpoppler126 Can be combined with the upcoming poppler transiton (see #1060019) > libvlc5 > libvlccore9 Change to vlc plugin ABI. This does not require a change of the binary package name. > g2clib Can be combined with the upcoming transition (see #1060077). Cheers -- Sebastian Ramacher
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 12:05:57PM +, Simon McVittie wrote: > On Fri, 05 Jan 2024 at 00:17:04 -0800, Steve Langasek wrote: > > - In multi-library packages, there is no reliable way to map from a set of > > headers in a dev package to specific shared libraries in a runtime library > > package that's not additionally computationally prohibitive; we therefore > > conservatively assume that if any headers from a source package show > > time_t ABI changes, all the runtime library packages from the source > > package are affected by the transition. > > 0 dbus-tests > Please ignore this specific binary package. The only public API/ABI > of src:dbus is in libdbus-1-3 + libdbus-1-dev, so analyzing those two > is enough. (dbus-tests accidentally contains one header file, but that's > a minor bug.) > libdbus-1-dev is widely depended-on, so I hope that taking src:dbus off > your list will avoid some unnecessary rebuilds? > > 0 gobject-introspection > Similarly the only public API/ABI of src:gobject-introspection is in > libgirepository1.0-dev, libgirepository-1.0-1, and (in experimental) > libgirepository-1.0-dev. gobject-introspection contains some source > and header files that are used by other packages' regression tests, > but they are not public ABI. Yes, sorry - the way my scripts are set up to do the analysis right now, these packages still show up in the `sorted-revdep-count` list but there are manual overrides mapping both of these to an empty set of runtime libraries. So you'll see they don't show up in the `runtime-libs` or `source-packages` lists, and none of the reverse-dependencies of libdbus or libgirepository are tagged for rebuild. https://salsa.debian.org/adrien-n/armhf-time_t/-/blob/c62a594236374469b2181e9c5578ed124b57c48c/packagelist.py#L304 -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Sebastian,
On Fri, Jan 05, 2024 at 06:34:38PM +0100, Sebastian Ramacher wrote:
> On 2024-01-05 00:23:00 -0800, Steve Langasek wrote:
> > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote:
> > > == Results ==
> > > The overall findings of this analysis are 1,745 "dev" packages which
> > > either are confirmed to have ABI changes or could not be checked;
> > > mapping to 2,154 runtime libraries (list attached) from 1,195 source
> > > packages (list attached) and 5,477 reverse-dependencies requiring
> > > no-change rebuilds (list attached). This is within the previously
> > > calculated range of "5300 to 5600", but there are a number of
> > > newly-identified packages that fail to compile and have a large number
> > > of reverse-dependencies. I will continue to work to identify
> > > false-positives here in the hopes of bringing this count down before
> > > pulling the trigger on an actual transition.
> > [...]
> > > In addition, Guillem pointed out that if there are libraries whose
> > > ABIs are lfs-sensitive but not time_t-sensitive, and either they
> > > themselves depend on libraries which are time_t-sensitive or they have
> > > reverse-dependencies that do, so they will also need to be included in
> > > the transition. I have identified a list of 53 packages in this
> > > category (list attached); these in turn have 174 additional
> > > reverse-dependencies that would need rebuilt (list attached).
> > I am also attaching here the dd-list output for the packages that will need
> > to be sourcefully NMUed for the transition, for your review.
> Why do the need sourceful NMUs if they just need to be rebuilt?
Sorry, if the original message hadn't been lost somewhere in the mail
system before being delivered to debian-devel (I've now tried to resend it),
this might have been clearer from context. Guillem points out the mail has
been delivered to debian-release, so you can read the whole thing there:
https://lists.debian.org/debian-release/2024/01/msg00033.html
Anyway, this is the list of source packages containing libraries whose ABI
will change. So the packages need to be renamed in order to expose the ABI
incompatibility to reverse-dependencies. That's not a no-change rebuild.
This is consistent with historical practice in Debian for
toolchain-triggered ABI changes ('g' for libc5->glibc; c102; c2; v5; ldbl)
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On 2024-01-05 00:23:00 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > == Results == > > > > The overall findings of this analysis are 1,745 "dev" packages which either > > are confirmed to have ABI changes or could not be checked; mapping to 2,154 > > runtime libraries (list attached) from 1,195 source packages (list attached) > > and 5,477 reverse-dependencies requiring no-change rebuilds (list attached). > > This is within the previously calculated range of "5300 to 5600", but there > > are a number of newly-identified packages that fail to compile and have a > > large number of reverse-dependencies. I will continue to work to identify > > false-positives here in the hopes of bringing this count down before pulling > > the trigger on an actual transition. > > [...] > > > In addition, Guillem pointed out that if there are libraries whose ABIs are > > lfs-sensitive but not time_t-sensitive, and either they themselves depend on > > libraries which are time_t-sensitive or they have reverse-dependencies that > > do, so they will also need to be included in the transition. I have > > identified a list of 53 packages in this category (list attached); these in > > turn have 174 additional reverse-dependencies that would need rebuilt (list > > attached). > > I am also attaching here the dd-list output for the packages that will need > to be sourcefully NMUed for the transition, for your review. Why do the need sourceful NMUs if they just need to be rebuilt? Cheers -- Sebastian Ramacher
Re: Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 05.01.24 um 09:17 schrieb Steve Langasek:
- Packages that could not be analyzed for whatever reason are still
assumed to have an ABI that's sensitive to time_t and have to be included
in the transition. Happily, due to improvements in this run of the number
of packages that could successfully be analyzed, the number of libraries
in this category has dropped from 1541 to 929, of which 69 have no
reverse-dependencies at all. The final list of these header packages that
could not be analyzed is attached, in case anyone still wants to identify
that a library on this list whose ABI is not affected by time_t and should
therefore be excluded from the transition. Note, however, that no effort
has been made to filter out dev packages from this list that come from
source packages containing OTHER dev packages that are known to have ABI
breakage; for a number of the packages listed, further analysis would not
change the outcome. (e.g. qt5-qmake failed to be analyzed, but
qtbase-opensource-src also ships qtbase5-private-dev which shows time_t
ABI sensitivity.)
[...]
My strawman proposal is to give this thread 2 weeks from today for feedback
and further refinement, and also to further reduce the number of
false-positives included in the transition. Then, starting on Jan 18:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
I think at that point in time one should know what breaks and whatnot.
Archive rebuild?
(Probably in stages)
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every
maintainer, but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid
immediately when tagged end of next week to not have this caught in the
transition. (see also below for the comment about new upstream versions
in experimental.)
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
Those also will need clear NEW if needed.
(And they might even FTBFS because the ABI did change and ABI-assuming
test break? Though I don't assume so for time_t, but if time is passed
around... I don't assume so, but...
At least that would be the case for libreoffice (and
libreoffice-dev-common is in the affected set, which means some stuff
relies on it...))
(Maybe even needing asm fixes)
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
- the sourceful NMUs of the libraries will be reuploaded to unstable
(without binaries, so that they can be promoted to testing without
additional uploads).
Please let me know of any problems with this plan.
Also a problem is that experimental also might already contain totally
unrelated updates like new upstream versions...
Regards,
Rene
Re: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi! [ It seems the original post didn't get through to debian-devel (yet?), I found it on debian-release though https://lists.debian.org/debian-release/2024/01/msg00033.html, you might want to repost it here though, so that it can be commented on properly? :) ] On Fri, 2024-01-05 at 00:23:00 -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 12:17:04AM -0800, Steve Langasek wrote: > > == Results == > > > > The overall findings of this analysis are 1,745 "dev" packages which either > > are confirmed to have ABI changes or could not be checked; mapping to 2,154 > > runtime libraries (list attached) from 1,195 source packages (list attached) > > and 5,477 reverse-dependencies requiring no-change rebuilds (list attached). > > This is within the previously calculated range of "5300 to 5600", but there > > are a number of newly-identified packages that fail to compile and have a > > large number of reverse-dependencies. I will continue to work to identify > > false-positives here in the hopes of bringing this count down before pulling > > the trigger on an actual transition. > Guillem Jover >libaio >libmd >liburing I checked these, and it looks like libmd and liburing are false-positives? * libmd uses AC_SYS_LARGEFILE, and on 32-bit arches it is already built with LFS, the problem is that the header exposes off_t which means the code linking against it needs to match its build flags, otherwise they would already be broken now. You might want to look into this as a potential pattern for other false-positives probably. (I should probably update upstream the .pc file to include the -D_FILE_OFFSET_BITS=64 flags if enabled, but I don't think the analysis used .pc files anyway.) * liburing is marked as LFS-sensitive, but that comes from inline functions using off_t which end up casting that into an u32 type, so I don't think this should affect the ABI. It is also forcibly built with -D_FILE_OFFSET_BITS=64 in the upstream build system (and with future=+lfs for good measure in the packaging, which I'll switch to abi=+lfs). Thanks, Guillem
