Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
> "Ansgar" == Ansgar writes: Ansgar> As far as I understand this approach will break any consumer Ansgar> on a library whose ABI changes to to the ABI changes Ansgar> introduced here unless the consumer is built with the flags Ansgar> from `dpkg-buildflags` (which would now define the ABI). Ansgar> Do we want this? I'm fairly sure we do not. Elsewhere in the thread, I saw one of the toolchain maintainers talking about what is involved in getting gcc to support the new flags. My suspicion is that handling things with dpkg-flags is more about moving quickly with the transition than a desired end state. Personally I'd consider it a release blocker if we don't get the compilers updated prior to the release. I think it would be reasonable to open a bug on the toolchain packages now talking about this. I think logistically it would not be desirable for those bugs to be RC at this time. Yes, if not fixed they will eventually need to be, but for example I don't think it would be desirable to block toolchain testing migrations on this issue at this time. And obviously we're not going to remove the toolchain from the release. --Sam Speaking with my armchair I have no hat in this non-hat on.
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, 6 Jan 2024 19:38:42 -0800 Steve Langasek wrote: > - dpkg will be uploaded to experimental with 64-bit time_t in the default > flags As far as I understand this approach will break any consumer on a library whose ABI changes to to the ABI changes introduced here unless the consumer is built with the flags from `dpkg-buildflags` (which would now define the ABI). In particular users could no longer just invoke `gcc` alone, but would have to also rely on `dpkg-buildflags` for any software they built on their own using runtime libraries shipped by Debian. It would work in some cases (multi-ABI libraries like libc6 or libraries not affected by the ABI change), but it is not reliable. Do we want this? It must at least be documented, probably in Debian Policy and GCC. This was asked on debian-devel@ multiple times, but there was no answer so far. Ansgar
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Colin,
On Tue, Jan 09, 2024 at 01:32:11PM +, Colin Watson wrote:
> On Mon, Jan 08, 2024 at 03:01:11PM -0800, Steve Langasek wrote:
> > On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote:
> > > On 05-01-2024 17:36, Rene Engelhard wrote:
> > > > Also a problem is that experimental also might already contain totally
> > > > unrelated updates like new upstream versions...
> > > I share this worry. Have you thought about how to handle the cases where
> > > you
> > > don't have experimental to upload to? How big is this problem?
> In the current situation, though, not having experimental available
> means that there's no opportunity for dumat to weigh in regarding
> usrmerge interactions, which seems problematic given Helmut's
> preliminary analysis.
I guess this is a reply to the bits of context I retained above rather than
directly a reply to my most immediate preceding message. In a separate
subthread, I laid out what I thought the process should be for handling the
transition of packages in that situation; but you are raising a separate
point.
Which parallels what Helmut had to say:
> Whenever you face files in aliased locations (other than systemd units),
> please go via experimental to let dumat judge your upload. Check the
> bookworm package for files in aliased locations, not the unstable one.
I have also had other out-of-band conversations with Helmut about the
overlapping issues between usrmerge and time_t, so this was generally
speaking on my radar although I didn't address it in my proposed transition
plan.
I think it is unrealistic to ask experimental to be otherwise quiesced of
transitions to ensure that we can stage all of the affected libraries in
experimental, and I also think that making false transitions for packages
that are ALREADY in experimental because of transitions would be
counterproductive. ("False" because if the library package has a different
soname between unstable and experimental, then renaming the runtime library
package in the *experimental* version is unnecessary because there's nothing
"in the wild" depending on that package name but with the old ABI for which
upgrade compatibility is of concern.)
My preferred path forward here would be, as Helmut suggests, to check which
libraries are in the intersection of (packages in experimental for which we
won't stage time_t uploads) and (packages that exist in bookworm and need
transitioning of aliased paths), and ensure that these packages are handled
carefully according to Helmut's own guidance. My optimistic expectation is
that the intersection will be null, or nearly; but in any case I will bring
data.
And my proposal for checking that set, since we're only talking about
runtime library packages, is to check whether any of the contents of these
packages in bookworm match ^/lib - as a runtime library package NOT matching
this pattern, but matching a pattern for one of the other aliased
directories, would be something ... special.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, Jan 08, 2024 at 03:01:11PM -0800, Steve Langasek wrote: > On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote: > > On 05-01-2024 17:36, Rene Engelhard wrote: > > > Also a problem is that experimental also might already contain totally > > > unrelated updates like new upstream versions... > > > I share this worry. Have you thought about how to handle the cases where you > > don't have experimental to upload to? How big is this problem? > > > Another worry, how will you provide the required changes to the maintainers > > of the packages? Via BTS? For those working on salsa: MR? Both? Something > > else? Obviously we should not end in the situation that a new upload goes > > back to the old name (or are the ftp-masters so keen on this transition that > > that won't happen? But what about newer versions with the old name already > > in experimental, conform the former worry?). I've seen NMU's being ignored > > by subsequent uploads by the maintainer, even when they fixed RC issues > > which were then reintroduced. > > I would intend to provide diffs via the BTS. This remains the standard > policy for NMUs in Debian per the Developer's Reference, and as far as I > know has worked effectively for all such previous ABI transitions. In the current situation, though, not having experimental available means that there's no opportunity for dumat to weigh in regarding usrmerge interactions, which seems problematic given Helmut's preliminary analysis. -- Colin Watson (he/him) [cjwat...@debian.org]
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, 08 Jan 2024 at 15:01:11 -0800, Steve Langasek wrote:
> If a maintainer ignores the NMU and drops the rename, they'll be introducing
> a new library transition again on 32-bit archs. Won't they also be caught
> again in binary NEW, for those packages that don't have the same runtime
> library package name in experimental?
To have a concrete example of this, I think you are saying:
- NMU of src:foo renames libfoo0 to libfoo0t64
- maintainer ignores NMU and uploads, effectively renaming libfoo0t64
back to libfoo0
- you want the maintainer's upload to get stuck in NEW
I am not a ftp team member, but if I understand NEW correctly, this
will only trigger a new trip through NEW if the ftp team have already
removed libfoo0 from the overrides file ("decrufting"), which is not
done immediately, only after libfoo0 has not been built by src:foo for
a little while.
If libfoo0 exists in testing and/or stable, I'm not sure whether that
prevents the ftp team's processes from removing it from the overrides file.
If it does, then a new, maintainer upload of libfoo0 will certainly not be
considered NEW, and the safety-catch that you are thinking of will not be
effective.
smcv
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 09:17:52PM +0100, Paul Gevers wrote: > Hi Steve, > On 05-01-2024 17:36, Rene Engelhard wrote: > > Also a problem is that experimental also might already contain totally > > unrelated updates like new upstream versions... > I share this worry. Have you thought about how to handle the cases where you > don't have experimental to upload to? How big is this problem? > Another worry, how will you provide the required changes to the maintainers > of the packages? Via BTS? For those working on salsa: MR? Both? Something > else? Obviously we should not end in the situation that a new upload goes > back to the old name (or are the ftp-masters so keen on this transition that > that won't happen? But what about newer versions with the old name already > in experimental, conform the former worry?). I've seen NMU's being ignored > by subsequent uploads by the maintainer, even when they fixed RC issues > which were then reintroduced. I would intend to provide diffs via the BTS. This remains the standard policy for NMUs in Debian per the Developer's Reference, and as far as I know has worked effectively for all such previous ABI transitions. I think it is reasonable to expect maintainers to pay attention to their bug mail as our defined Debian process. I don't think it would be reasonable to expect people working on cross-archive toolchain transitions to cater to individual maintainer preference in this regard. If a maintainer ignores the NMU and drops the rename, they'll be introducing a new library transition again on 32-bit archs. Won't they also be caught again in binary NEW, for those packages that don't have the same runtime library package name in experimental? It seems to me there's ample opportunity to catch such mistakes if they happen. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Mon, Jan 08, 2024 at 04:12:42PM +0100, Julian Andres Klode wrote: > > - once these packages have all cleared binary NEW, the new dpkg defaults > > will be uploaded to unstable > What happened to the plan to workaround this by doing dak database > shenanigans prior to uploading to avoid binary-NEW altogether, that > I chatted about with mhy/#debian-ftp and you? Did that not work out? That wasn't called out here but is part of the implementation details of the plan. The intention is still to go through binary NEW in experimental before copying to unstable, because it will take time to get all the binary uploads done (longer than it will take to get the sourceful uploads to unstable done), so it's better to stage in experimental to minimize the window in unstable when uploads can be broken. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hey Steve,
On Sat, Jan 06, 2024 at 07:38:42PM -0800, Steve Langasek wrote:
> On Sat, Jan 06, 2024 at 09:25:52AM +0100, Rene Engelhard wrote:
> > Hi,
>
> > Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > > default
> > > > > flags
> > > [...]
> > > What about the suggestion to not push changes to experimental for packages
> > > that already have new versions in experimental, and do the binary package
> > > renames in unstable instead, leaving the package in experimental alone?
>
> > How does that play together with the needed dpkg only in experimental?
>
> > You can't build stuff for unstable involving experimental packages (except
> > manually with binary upload, which would block testing migration)
>
> The ordering here would be:
>
> - dpkg will be uploaded to experimental with 64-bit time_t in the default
> flags
>
> - the source packages which need an ABI change
> ("source-packages"+"lfs-and-depends-time_t") and do not already have
> versions in experimental, will have sourceful NMUs to experimental with
> the new binary package names in order to clear binary NEW, in coordination
>
> - once these packages have all cleared binary NEW, the new dpkg defaults
> will be uploaded to unstable
What happened to the plan to workaround this by doing dak database
shenanigans prior to uploading to avoid binary-NEW altogether, that
I chatted about with mhy/#debian-ftp and you? Did that not work out?
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
signature.asc
Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 07.01.24 um 04:38 schrieb Steve Langasek:
The ordering here would be:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t") and do not already have
versions in experimental, will have sourceful NMUs to experimental with
the new binary package names in order to clear binary NEW, in coordination
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
And in the meanwhile in experimental it will be built with the old
time_t on other architectures.
Since the new dpkg won't be picked up.
Not in the experimental builders unstable+experimental chroots which
only install experimental packages when Build-Depends: need them.
(For an undefined time given how much the packages later in the chain
wait in NEW)
Especially on armhf which is affected. Or will you do the source NMUs on
armhf/i386? (For some packages where some features are disabled on 32bit
this is probably not a good idea)
Regards,
Rene
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 07.01.24 um 02:01 schrieb Steve Langasek:
If you say you are going to fix eventual breakage (and not ignoring
the test
results!) and if that means fixing asm on all affected archs, then it's OK
:)
Well, yes; though I hope we would see some help from e.g. arm porters if
there were actually breakage of this nature.
Hope doesn't help when it got to the actual problem and this doesn't happen.
Alternatively, maybe it would
be better to stop shipping libreoffice on 32-bit archs, in that case? I'm
I'd like to avoid this. Getting rid of i386 is fine, noone needs it,
armhf is a bit different.
(rpis etc - even though they could run arm64, but..)
not sure how usable libreoffice is these days on such archs.
popcon.debian.org doesn't appear to have the granularity to tell us if there
actually *are* users; and with only 615 armhf popcon reports (vs 215,562 on
amd64) we don't exactly have a statistically significant sample there.
Only recently I got
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1059158. Maybe UI is
not used much, but I can imagine people using it in paperless-ngx or
other stuff.
I don't really want to bastardize those uses.
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every maintainer,
but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
when tagged end of next week to not have this caught in the transition. (see
also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
If at all - *both*. At the same time.
Most of these packages that are staged in experimental are going to be there
because of library transitions...
And ignore those who use experimental as a testbed of major new
upstreams? Doesn't sound good.
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
(which incidentially contains libreoffice)
Ah! These are packages that have been omitted from the analysis because
they've been manually identified as packages that don't have C or
C++-compatible header files related to a shared library, and therefore even
if they do need updated for 64-bit time_t, they are not affected by CPPFLAGS
and are not part of this transition. (I am not 100% sure this is accurate
for ObjC; in any case ObjC headers are impossible to analyze using
abi-compliance-checker so if ObjC libraries are possibly affected, someone
would have to figure out what to do with them.)
I have no idea on how you indentified it In the libreoffice case that is
not true.
libreoffice-dev-(common) *does* contain C/C++ header files.
And most of them *do* correspond to the libuno* shared packages.
Just that they are not split per library because that wouldn't really
make sense since you need any of them anyway.
And I definitely see
e.g. /usr/include/libreoffice/osl/time.h (libuno-sal3)
No idea whether it actually will be broken by the change, but...
I'm not sure precisely why Adrien found it necessary/useful to add
libreoffice-dev to this exclude list,
Probably because he didn't look at the whole but just at the package
but I can confirm that it's
reasonable, as this particular package contains only a single header file
with #define's and no function prototypes; so in effect it has been manually
analyzed and determined to be unaffected.
But libreoffice-dev-common not (on which libreoffice-dev depends) which
contains all the arch-indep headers. Just libreoffice-dev contains one
header diferent between archs.
Regards,
Rene
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 09:25:52AM +0100, Rene Engelhard wrote:
> Hi,
> Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > default
> > > > flags
> > [...]
> > What about the suggestion to not push changes to experimental for packages
> > that already have new versions in experimental, and do the binary package
> > renames in unstable instead, leaving the package in experimental alone?
> How does that play together with the needed dpkg only in experimental?
> You can't build stuff for unstable involving experimental packages (except
> manually with binary upload, which would block testing migration)
The ordering here would be:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t") and do not already have
versions in experimental, will have sourceful NMUs to experimental with
the new binary package names in order to clear binary NEW, in coordination
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
- source packages which need an ABI change but already have versions in
experimental will be uploaded to unstable, with binaries, to clear binary
NEW
- sourceful NMUs of all the libraries will be reuploaded to unstable
(without binaries, so that they can be promoted to testing without
additional uploads).
- perl will also get a sourceful upload, to manually handle 'perlapi'
Provides.
- binNMUs will be scheduled for all of the reverse-dependencies.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Sat, Jan 06, 2024 at 09:07:15AM +0100, Rene Engelhard wrote:
> Am 06.01.24 um 06:51 schrieb Steve Langasek:
> > > > - dpkg will be uploaded to experimental with 64-bit time_t in the
> > > > default
> > > > flags
> > > I think at that point in time one should know what breaks and whatnot.
> > > Archive rebuild?
> > > (Probably in stages)
> > What kind of breakage are you looking to avoid here?
> build failures/test failures.
> > As mentioned in other points in the thread, regressions as a result of
> > this change should be rare and easy to fix. I do not think it's a good
> > use of time / CPU power to do test rebuilds for this instead of just
> > landing the transition.
> Here especially LibreOffices bridge code worries me.
> That one is tied to ABI and calling conventions. I don't see time_t
> mentioned there but "just" in the public libraries (sal), but I am not sure
> what making a type bigger will cause.
> (And since already
> - i386 needs gcc-12 to build since otherwise the test fails
> - armhf (and other archs like ppc64el and s390x) need Java disabled[1] -
> without any help from any porter to prevent this - I *do* want to avoid
> breakage here.
> If you say you are going to fix eventual breakage (and not ignoring the test
> results!) and if that means fixing asm on all affected archs, then it's OK
> :)
Well, yes; though I hope we would see some help from e.g. arm porters if
there were actually breakage of this nature. Alternatively, maybe it would
be better to stop shipping libreoffice on 32-bit archs, in that case? I'm
not sure how usable libreoffice is these days on such archs.
popcon.debian.org doesn't appear to have the granularity to tell us if there
actually *are* users; and with only 615 armhf popcon reports (vs 215,562 on
amd64) we don't exactly have a statistically significant sample there.
> > > > - the source packages which need an ABI change
> > > > ("source-packages"+"lfs-and-depends-time_t" will have sourceful
> > > > NMUs to
> > > I get that you probably want NMUs for not needing to ping every
> > > maintainer,
> > > but this is bad.
> > > That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
> > > when tagged end of next week to not have this caught in the transition.
> > > (see
> > > also below for the comment about new upstream versions in experimental.)
> > What about the suggestion to not push changes to experimental for packages
> > that already have new versions in experimental, and do the binary package
> > renames in unstable instead, leaving the package in experimental alone?
> If at all - *both*. At the same time.
Most of these packages that are staged in experimental are going to be there
because of library transitions... so I think we definitely don't want to be
renaming those binary packages, they should just drop the t64 suffix when
they move to unstable.
> But yes, that could work.
> (In this specific case I an worrying that the transition will take some
> time, and that I am stuck with 7.6.x instead of uploading 24.2 when it is
> released.)
Ok. I don't think there would be any need for you to wait before uploading
24.2. It might have to wait for time_t for testing migration, but I don't think that should really be long.
> > > > experimental with the new binary package names in order to clear
> > > > binary
> > > > NEW, in coordination
> > > And what about skipped ones? When will those be tried?
> > What do you mean here by "skipped ones"?
> https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
> (which incidentially contains libreoffice)
Ah! These are packages that have been omitted from the analysis because
they've been manually identified as packages that don't have C or
C++-compatible header files related to a shared library, and therefore even
if they do need updated for 64-bit time_t, they are not affected by CPPFLAGS
and are not part of this transition. (I am not 100% sure this is accurate
for ObjC; in any case ObjC headers are impossible to analyze using
abi-compliance-checker so if ObjC libraries are possibly affected, someone
would have to figure out what to do with them.)
I'm not sure precisely why Adrien found it necessary/useful to add
libreoffice-dev to this exclude list, but I can confirm that it's
reasonable, as this particular package contains only a single header file
with #define's and no function prototypes; so in effect it has been manually
analyzed and determined to be unaffected.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi, Am 06.01.24 um 06:51 schrieb Steve Langasek: - dpkg will be uploaded to experimental with 64-bit time_t in the default flags [...] What about the suggestion to not push changes to experimental for packages that already have new versions in experimental, and do the binary package renames in unstable instead, leaving the package in experimental alone? How does that play together with the needed dpkg only in experimental? You can't build stuff for unstable involving experimental packages (except manually with binary upload, which would block testing migration) Regards, Rene
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve,
Am 06.01.24 um 06:51 schrieb Steve Langasek:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
I think at that point in time one should know what breaks and whatnot.
Archive rebuild?
(Probably in stages)
What kind of breakage are you looking to avoid here? As mentioned in other
build failures/test failures.
points in the thread, regressions as a result of this change should be rare
and easy to fix. I do not think it's a good use of time / CPU power to do
test rebuilds for this instead of just landing the transition.
Here especially LibreOffices bridge code worries me.
That one is tied to ABI and calling conventions. I don't see time_t
mentioned there but "just" in the public libraries (sal), but I am not
sure what making a type bigger will cause.
(And since already
- i386 needs gcc-12 to build since otherwise the test fails
- armhf (and other archs like ppc64el and s390x) need Java disabled[1]
- without any help from any porter to prevent this - I *do* want to
avoid breakage here.
If you say you are going to fix eventual breakage (and not ignoring the
test results!) and if that means fixing asm on all affected archs, then
it's OK :)
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every maintainer,
but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
when tagged end of next week to not have this caught in the transition. (see
also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
If at all - *both*. At the same time.
But yes, that could work.
(In this specific case I an worrying that the transition will take some
time, and that I am stuck with 7.6.x instead of uploading 24.2 when it
is released.)
libreoffice libs only have one reverse-dep, zemberek-ooo; so the risk of
entanglement here is small anyway.
Except in poppler etc. transitions.. But yeah, nothing really uses the
C++ libs nowadays.
I think the above proposal, to skip packages already in experimental from
the set of uploads to experimental, would address your concern. It's not as
if there is going to be any time that it's ok to tell maintainers they can't
use experimental at all because we're doing this transition.
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/results_skipped.txt
(which incidentially contains libreoffice)
Regards,
Rene
[1] Ubuntu just ignores the test failures. No, not an option.
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 05:36:10PM +0100, Rene Engelhard wrote:
> > My strawman proposal is to give this thread 2 weeks from today for feedback
> > and further refinement, and also to further reduce the number of
> > false-positives included in the transition. Then, starting on Jan 18:
> > - dpkg will be uploaded to experimental with 64-bit time_t in the default
> >flags
> I think at that point in time one should know what breaks and whatnot.
> Archive rebuild?
> (Probably in stages)
What kind of breakage are you looking to avoid here? As mentioned in other
points in the thread, regressions as a result of this change should be rare
and easy to fix. I do not think it's a good use of time / CPU power to do
test rebuilds for this instead of just landing the transition.
> > - the source packages which need an ABI change
> >("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
> I get that you probably want NMUs for not needing to ping every maintainer,
> but this is bad.
> That e.g. would cause me to upload libreoffice 24.2 rc2 to sid immediately
> when tagged end of next week to not have this caught in the transition. (see
> also below for the comment about new upstream versions in experimental.)
What about the suggestion to not push changes to experimental for packages
that already have new versions in experimental, and do the binary package
renames in unstable instead, leaving the package in experimental alone?
libreoffice libs only have one reverse-dep, zemberek-ooo; so the risk of
entanglement here is small anyway.
I think the above proposal, to skip packages already in experimental from
the set of uploads to experimental, would address your concern. It's not as
if there is going to be any time that it's ok to tell maintainers they can't
use experimental at all because we're doing this transition.
> >experimental with the new binary package names in order to clear binary
> >NEW, in coordination
> And what about skipped ones? When will those be tried?
What do you mean here by "skipped ones"?
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer https://www.debian.org/
slanga...@ubuntu.com vor...@debian.org
signature.asc
Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve, On 05-01-2024 17:36, Rene Engelhard wrote: Also a problem is that experimental also might already contain totally unrelated updates like new upstream versions... I share this worry. Have you thought about how to handle the cases where you don't have experimental to upload to? How big is this problem? Another worry, how will you provide the required changes to the maintainers of the packages? Via BTS? For those working on salsa: MR? Both? Something else? Obviously we should not end in the situation that a new upload goes back to the old name (or are the ftp-masters so keen on this transition that that won't happen? But what about newer versions with the old name already in experimental, conform the former worry?). I've seen NMU's being ignored by subsequent uploads by the maintainer, even when they fixed RC issues which were then reintroduced. Paul OpenPGP_signature.asc Description: OpenPGP digital signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi Steve > - perl will also get a sourceful upload, to manually handle 'perlapi' > Provides. Can we combine this one with the upcoming perl transition? See #1055955. Here are some more comments on individual packages. > 22 libobs-dev That's a change to the plugin ABI only. > 14 gnuradio gnuradio bump its SONAME every other month. > 9 libopenmpt-dev Seems to be a false positive. > 9 libogre-1.9-dev > 9 libgirara-dev Why is this one on the list? > 5 gcc-10-plugin-dev Can be skipped, not part of testing. Should be RMed from the archive instead. > 4 llvm-15-dev llvm-toolchain-15 is scheduled for removal. Reverse dependencies should get an RC bug instead. > 4 libspatialaudio-dev Why is this package on the list? > 3 libdvbpsi-dev Seems to be a false positive > libavutil58 av_timegm is not used by any package in the archive. I'd skip ffmpeg and it's libraries. > libpoppler-cpp0v5 > libpoppler-glib8 > libpoppler-qt5-1 > libpoppler-qt6-3 > libpoppler126 Can be combined with the upcoming poppler transiton (see #1060019) > libvlc5 > libvlccore9 Change to vlc plugin ABI. This does not require a change of the binary package name. > g2clib Can be combined with the upcoming transition (see #1060077). Cheers -- Sebastian Ramacher
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, Jan 05, 2024 at 12:05:57PM +, Simon McVittie wrote: > On Fri, 05 Jan 2024 at 00:17:04 -0800, Steve Langasek wrote: > > - In multi-library packages, there is no reliable way to map from a set of > > headers in a dev package to specific shared libraries in a runtime library > > package that's not additionally computationally prohibitive; we therefore > > conservatively assume that if any headers from a source package show > > time_t ABI changes, all the runtime library packages from the source > > package are affected by the transition. > > 0 dbus-tests > Please ignore this specific binary package. The only public API/ABI > of src:dbus is in libdbus-1-3 + libdbus-1-dev, so analyzing those two > is enough. (dbus-tests accidentally contains one header file, but that's > a minor bug.) > libdbus-1-dev is widely depended-on, so I hope that taking src:dbus off > your list will avoid some unnecessary rebuilds? > > 0 gobject-introspection > Similarly the only public API/ABI of src:gobject-introspection is in > libgirepository1.0-dev, libgirepository-1.0-1, and (in experimental) > libgirepository-1.0-dev. gobject-introspection contains some source > and header files that are used by other packages' regression tests, > but they are not public ABI. Yes, sorry - the way my scripts are set up to do the analysis right now, these packages still show up in the `sorted-revdep-count` list but there are manual overrides mapping both of these to an empty set of runtime libraries. So you'll see they don't show up in the `runtime-libs` or `source-packages` lists, and none of the reverse-dependencies of libdbus or libgirepository are tagged for rebuild. https://salsa.debian.org/adrien-n/armhf-time_t/-/blob/c62a594236374469b2181e9c5578ed124b57c48c/packagelist.py#L304 -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developer https://www.debian.org/ slanga...@ubuntu.com vor...@debian.org signature.asc Description: PGP signature
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi,
Am 05.01.24 um 09:17 schrieb Steve Langasek:
- Packages that could not be analyzed for whatever reason are still
assumed to have an ABI that's sensitive to time_t and have to be included
in the transition. Happily, due to improvements in this run of the number
of packages that could successfully be analyzed, the number of libraries
in this category has dropped from 1541 to 929, of which 69 have no
reverse-dependencies at all. The final list of these header packages that
could not be analyzed is attached, in case anyone still wants to identify
that a library on this list whose ABI is not affected by time_t and should
therefore be excluded from the transition. Note, however, that no effort
has been made to filter out dev packages from this list that come from
source packages containing OTHER dev packages that are known to have ABI
breakage; for a number of the packages listed, further analysis would not
change the outcome. (e.g. qt5-qmake failed to be analyzed, but
qtbase-opensource-src also ships qtbase5-private-dev which shows time_t
ABI sensitivity.)
[...]
My strawman proposal is to give this thread 2 weeks from today for feedback
and further refinement, and also to further reduce the number of
false-positives included in the transition. Then, starting on Jan 18:
- dpkg will be uploaded to experimental with 64-bit time_t in the default
flags
I think at that point in time one should know what breaks and whatnot.
Archive rebuild?
(Probably in stages)
- the source packages which need an ABI change
("source-packages"+"lfs-and-depends-time_t" will have sourceful NMUs to
I get that you probably want NMUs for not needing to ping every
maintainer, but this is bad.
That e.g. would cause me to upload libreoffice 24.2 rc2 to sid
immediately when tagged end of next week to not have this caught in the
transition. (see also below for the comment about new upstream versions
in experimental.)
experimental with the new binary package names in order to clear binary
NEW, in coordination
And what about skipped ones? When will those be tried?
Those also will need clear NEW if needed.
(And they might even FTBFS because the ABI did change and ABI-assuming
test break? Though I don't assume so for time_t, but if time is passed
around... I don't assume so, but...
At least that would be the case for libreoffice (and
libreoffice-dev-common is in the affected set, which means some stuff
relies on it...))
(Maybe even needing asm fixes)
- once these packages have all cleared binary NEW, the new dpkg defaults
will be uploaded to unstable
- the sourceful NMUs of the libraries will be reuploaded to unstable
(without binaries, so that they can be promoted to testing without
additional uploads).
Please let me know of any problems with this plan.
Also a problem is that experimental also might already contain totally
unrelated updates like new upstream versions...
Regards,
Rene
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
On Fri, 05 Jan 2024 at 00:17:04 -0800, Steve Langasek wrote: > - In multi-library packages, there is no reliable way to map from a set of > headers in a dev package to specific shared libraries in a runtime library > package that's not additionally computationally prohibitive; we therefore > conservatively assume that if any headers from a source package show > time_t ABI changes, all the runtime library packages from the source > package are affected by the transition. > 0 dbus-tests Please ignore this specific binary package. The only public API/ABI of src:dbus is in libdbus-1-3 + libdbus-1-dev, so analyzing those two is enough. (dbus-tests accidentally contains one header file, but that's a minor bug.) libdbus-1-dev is widely depended-on, so I hope that taking src:dbus off your list will avoid some unnecessary rebuilds? > 0 gobject-introspection Similarly the only public API/ABI of src:gobject-introspection is in libgirepository1.0-dev, libgirepository-1.0-1, and (in experimental) libgirepository-1.0-dev. gobject-introspection contains some source and header files that are used by other packages' regression tests, but they are not public ABI. smcv
Bug#1036884: 64-bit time_t: updated archive analysis, proposed transition plan with timeline
Hi folks, Once again, coming back around to the question of an archive-wide migration for time_t[0]. We have a clear path forward on the toolchain side (https://bugs.debian.org/1037136), and we have an updated ABI analysis done against current Debian unstable as of 2023-12-18, superseding the previous preliminary analysis we had done on Ubuntu mantic. This re-analysis took a while to get done in large part because the earlier mailing list thread revealed gaps in the identification of "dev" packages[1], so we spent some time fixing this to ensure we had proper archive coverage. Output files from the new analysis can be found here: https://adrien.dcln.fr/misc/armhf-time_t/2023-12-18/ == Description of methodology == Before going into details of the plan I'm proposing for the migration, I want to surface to the list the methodology used for the analysis, so that people have an opportunity to identify any errors. - Packages are identified as "dev" packages if, according to Contents-armhf in the archive, they ship files with an extension of .h, .hpp, .hxx, or .hh in any directory outside of /usr/share/doc. - Packages are identified as "library" packages if, according to Contents, they ship files with an extension of .so or match the pattern *.so.*. - Source packages are candidates for ABI analysis as part of this transition if at least one of their binary packages is a "dev" package, at least one is a "library" package, and at least one ships a shlibs control file. - This means packages that provide plugin APIs are excluded from the transition (they couldn't be automated anyway), but might still have ABI breakage that will require handling. (ex: apache2-dev; but it looks like perl did get picked up correctly by way of libperl5.36) - In multi-library packages, there is no reliable way to map from a set of headers in a dev package to specific shared libraries in a runtime library package that's not additionally computationally prohibitive; we therefore conservatively assume that if any headers from a source package show time_t ABI changes, all the runtime library packages from the source package are affected by the transition. This seems sensible in general, but er, in the pathological case we have Source: zlib that now ships both zlib1g-dev and libminizip-dev, zlib1g-dev is confirmed to not be ABI breaking, libminizip-dev has failed to analyze, and we don't want to force a transition for zlib1g because of libminizip (!). Current plan is to simply special case zlib1g, but there could be other problem packages we haven't identified. - Packages that could not be analyzed for whatever reason are still assumed to have an ABI that's sensitive to time_t and have to be included in the transition. Happily, due to improvements in this run of the number of packages that could successfully be analyzed, the number of libraries in this category has dropped from 1541 to 929, of which 69 have no reverse-dependencies at all. The final list of these header packages that could not be analyzed is attached, in case anyone still wants to identify that a library on this list whose ABI is not affected by time_t and should therefore be excluded from the transition. Note, however, that no effort has been made to filter out dev packages from this list that come from source packages containing OTHER dev packages that are known to have ABI breakage; for a number of the packages listed, further analysis would not change the outcome. (e.g. qt5-qmake failed to be analyzed, but qtbase-opensource-src also ships qtbase5-private-dev which shows time_t ABI sensitivity.) == Results == The overall findings of this analysis are 1,745 "dev" packages which either are confirmed to have ABI changes or could not be checked; mapping to 2,154 runtime libraries (list attached) from 1,195 source packages (list attached) and 5,477 reverse-dependencies requiring no-change rebuilds (list attached). This is within the previously calculated range of "5300 to 5600", but there are a number of newly-identified packages that fail to compile and have a large number of reverse-dependencies. I will continue to work to identify false-positives here in the hopes of bringing this count down before pulling the trigger on an actual transition. My understanding is that this will also result in a perl ABI transition, separately from the libperl ABI transition; since this dependency isn't calculated via shlibs it was not automatically included in this analysis. This seems to be at most about 600 additional packages to be included in the transition. In addition, Guillem pointed out that if there are libraries whose ABIs are lfs-sensitive but not time_t-sensitive, and either they themselves depend on libraries which are time_t-sensitive or they have reverse-dependencies that do, so they will also need to be included in the transition. I have identified a list of 53 packages in this
