Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, Jun 17, 2024 at 02:24:18PM +0100, Luca Boccassi wrote: > > IOW, I can't think of any situation where mapping VERSION_CODENAME to > > osVersion would be a problem, and it's more stable. I would be happy > > to > > stand corrected, though! > > Please don't - these notes are there to be machine-readable, so the > format needs to be sensible and common, so unnecessary deviations will > be detrimental to the effort. This is sourced directly from os-release, > which as Julian said is set at the beginning. May I suggest that you change the spec then please? As written, it looks like distributions are free to choose how to populate these fields, rather than it being a mandatory mapping from /etc/os-release (given that we provide that). Or, if it's just that you expect the version to be orderable, perhaps specify that? > The optimal thing to me would sound like doing an archive-wide > (ignoring arch:all only) rebuild once os-release is changed at the > beginning of the cycle (and again if it were to happen later, too), so > that the change can be picked up. This also solves the problem of > packages not being rebuilt. And it seems like a desirable thing as > you'll be using the latest compiler, with the latest hardening fixes, > and so on, so it would be a win/win. This would be up to the Ubuntu Release Team to organise, but it would be a very significant change in our development workflow and I don't think it would be practical. Notably, Debian doesn't do this either. > If this is not possible or desirable for any reason, then please just > omit the field entirely. Anything parsing it needs to expect this to be > missing anyway, for the rolling distros use case. Ack. I was concerned before that omitting it would cause more problems for ecosystem tooling, but given that you're saying it's fine, that also sounds fine to me. It might be worth you also adding this specific recommendation to the spec for distributions that carry forward builds across releases such as Debian and Ubuntu. Robie signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
> IOW, I can't think of any situation where mapping VERSION_CODENAME to > osVersion would be a problem, and it's more stable. I would be happy > to > stand corrected, though! Please don't - these notes are there to be machine-readable, so the format needs to be sensible and common, so unnecessary deviations will be detrimental to the effort. This is sourced directly from os-release, which as Julian said is set at the beginning. The optimal thing to me would sound like doing an archive-wide (ignoring arch:all only) rebuild once os-release is changed at the beginning of the cycle (and again if it were to happen later, too), so that the change can be picked up. This also solves the problem of packages not being rebuilt. And it seems like a desirable thing as you'll be using the latest compiler, with the latest hardening fixes, and so on, so it would be a win/win. If this is not possible or desirable for any reason, then please just omit the field entirely. Anything parsing it needs to expect this to be missing anyway, for the rolling distros use case. -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, 17 Jun 2024 at 23:32, Michael Hudson-Doyle wrote: > > On Sat, 15 Jun 2024 at 10:46, Benjamin Drung wrote: >> >> Hi everyone, > > > Hi Benjamin, > >> >> I just uploaded dpkg 1.22.6ubuntu11 to Ubuntu oracular. This version >> enables ELF packaging metadata via dpkg-buildflags by default. ELF >> objects will record the spec https://systemd.io/ELF_PACKAGE_METADATA/ > > > So these changes break a few things, because they assume that if the > environment is affected by dpkg-buildflags, it is also affected by code that > is only part of running dpkg-buildpackage itself. There are a few ways that > this can go wrong: > > 1) running "./debian/rules binary" or whatever instead of dpkg-buildpackage > (which afaict is still the interface to package building that's dictated by > policy) > 2) code like that in cargo-auto-test that sources dpkg-buildflags output to > get behaviour close to a package build without actually doing a package build. > > I don't know what the ideal change for this is. I guess the code in > add_build_flags could check that the DPKG_BUILDPACKAGE_* variables the spec > files need are defined before including the spec files in LDFLAGS? That could be possible, but it would hide all issues. So first you should make a decision: do you want to track and fix all of these, or do you want to accept shipping an undetermined amount of packages that are lacking the notes? If the former, then it would probably be better to add an explicit override. That way it can be searched, so you can know from sources only, without needing to unpack and inspect binary packages, how many packages need fixing. -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Tue, 2024-06-18 at 10:32 +1200, Michael Hudson-Doyle wrote: > On Sat, 15 Jun 2024 at 10:46, Benjamin Drung wrote: > > Hi everyone, > > > > > Hi Benjamin, > > > I just uploaded dpkg 1.22.6ubuntu11 to Ubuntu oracular. This version > > enables ELF packaging metadata via dpkg-buildflags by default. ELF > > objects will record the spec https://systemd.io/ELF_PACKAGE_METADATA/ > > > > > So these changes break a few things, because they assume that if the > environment is affected by dpkg-buildflags, it is also affected by code > that is only part of running dpkg-buildpackage itself. There are a few > ways that this can go wrong: > > 1) running "./debian/rules binary" or whatever instead of dpkg-buildpackage >(which afaict is still the interface to package building that's dictated >by policy) > 2) code like that in cargo-auto-test that sources dpkg-buildflags output to >get behaviour close to a package build without actually doing a package >build. > > I don't know what the ideal change for this is. I guess the code in > add_build_flags could check that the DPKG_BUILDPACKAGE_* variables the spec > files need are defined before including the spec files in LDFLAGS? This issue is tracked in [1] now and I am working on a solution (set the variables in case they are not present in the environment). [1] https://bugs.launchpad.net/ubuntu/+source/dpkg/+bug/2069722 -- Benjamin Drung Debian & Ubuntu Developer -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Sat, 15 Jun 2024 at 10:46, Benjamin Drung wrote: > Hi everyone, > Hi Benjamin, > I just uploaded dpkg 1.22.6ubuntu11 to Ubuntu oracular. This version > enables ELF packaging metadata via dpkg-buildflags by default. ELF > objects will record the spec https://systemd.io/ELF_PACKAGE_METADATA/ So these changes break a few things, because they assume that if the environment is affected by dpkg-buildflags, it is also affected by code that is only part of running dpkg-buildpackage itself. There are a few ways that this can go wrong: 1) running "./debian/rules binary" or whatever instead of dpkg-buildpackage (which afaict is still the interface to package building that's dictated by policy) 2) code like that in cargo-auto-test that sources dpkg-buildflags output to get behaviour close to a package build without actually doing a package build. I don't know what the ideal change for this is. I guess the code in add_build_flags could check that the DPKG_BUILDPACKAGE_* variables the spec files need are defined before including the spec files in LDFLAGS? Cheers, mwh -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, Jun 17, 2024 at 01:03:56PM +0100, Robie Basak wrote: > In > Ubuntu's case (and Debian's), that's possible with the name, version and > os fields only, without ambiguity. It just occurred to me that in Debian's case there are binNMUs which would require reference to a different set of build artifacts. But I guess that would have to be resolved by adding more fields, which the spec allows. signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, Jun 17, 2024 at 01:37:22PM +0200, Benjamin Drung wrote: > The implementation maps VERSION_ID from /etc/os-release to the osVersion > key. Do you suggest to use VERSION_CODENAME from /etc/os-release > instead? Yes. That seems like the closest thing we have that would be suitable. > The spec [1] says about the osVersion description "The OS version, > typically corresponding to VERSION_ID in os-release". I am not sure if > we should osVersion in that case or introduce a new osVersionCodename > key. > > [1] https://systemd.io/ELF_PACKAGE_METADATA/ It seems like the spec hasn't been written with consideration in mind for distributions whose eventual release version isn't strictly known at binary build time. The spec specifies osVersion as only "typically corresponding to VERSION_ID" so this isn't a hard requirement. This seems like an appropriate place to deviate from the recommendation to try to fit the original intent. The "Motivation" section describes wanting to map distribution-provided binaries back to their build information (eg. via debuginfo). In Ubuntu's case (and Debian's), that's possible with the name, version and os fields only, without ambiguity. If general tooling uses osVersion, I guess that we just need to be consistent to avoid causing problems then? I don't see anything else under "Motivation" for which this wouldn't work. I suppose not shipping osVersion would also work but has a greater chance of failing in some tool or other, given how it's described in the spec. Using osVersionCodename is sort of equivalent to this option from this perspective, since it'd be the same as "osVersion missing" from a generic tool's perspective. It might also be worth noting that not all binaries are rebuilt every release, and so osVersion for ELF binaries would not correspond to /etc/os-release on a given user's system in the general anyway. For Debian and Ubuntu this limits the usefulness of this field. As above, for build artifacts we don't need it to be unambigious (but for other distros that may not be true). IOW, I can't think of any situation where mapping VERSION_CODENAME to osVersion would be a problem, and it's more stable. I would be happy to stand corrected, though! Robie signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, 2024-06-17 at 12:05 +0100, Robie Basak wrote: > On Mon, Jun 17, 2024 at 12:56:12PM +0200, Julian Andres Klode wrote: > > It is communicated directly at archive opening, when we upload the > > base-files containing the os-release saying that: > > > > $ grep 24.10 /etc/os-release > > VERSION_ID="24.10" > > VERSION="24.10 (Oracular Oriole)" > > > > I believe it's fair to say at this point that even if we did > > move a release, we'd not change the version anymore because > > there will have been extensive communication over a multitude > > of channels already using the version; third-party vendors > > will have done testing against that specific version > > string, etc. > > In this situation I'd be in favour of changing /etc/os-release (and > distro-info-data etc) to match whatever we end up calling the release. > Otherwise you'd end up with marketing materials saying one thing and > /etc/os-release saying another, which would lead to no end of user > support enquiries asking how to upgrade from one to the other. > > Before release, anything in the release is subject to change, so we > wouldn't be breaking any promises or expectations. Some users would have > to adapt in very specific cases, but that's what they would have signed > up for by choosing to rely on development release. The implementation maps VERSION_ID from /etc/os-release to the osVersion key. Do you suggest to use VERSION_CODENAME from /etc/os-release instead? The spec [1] says about the osVersion description "The OS version, typically corresponding to VERSION_ID in os-release". I am not sure if we should osVersion in that case or introduce a new osVersionCodename key. [1] https://systemd.io/ELF_PACKAGE_METADATA/ -- Benjamin Drung Debian & Ubuntu Developer -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, Jun 17, 2024 at 12:56:12PM +0200, Julian Andres Klode wrote: > It is communicated directly at archive opening, when we upload the > base-files containing the os-release saying that: > > $ grep 24.10 /etc/os-release > VERSION_ID="24.10" > VERSION="24.10 (Oracular Oriole)" > > I believe it's fair to say at this point that even if we did > move a release, we'd not change the version anymore because > there will have been extensive communication over a multitude > of channels already using the version; third-party vendors > will have done testing against that specific version > string, etc. In this situation I'd be in favour of changing /etc/os-release (and distro-info-data etc) to match whatever we end up calling the release. Otherwise you'd end up with marketing materials saying one thing and /etc/os-release saying another, which would lead to no end of user support enquiries asking how to upgrade from one to the other. Before release, anything in the release is subject to change, so we wouldn't be breaking any promises or expectations. Some users would have to adapt in very specific cases, but that's what they would have signed up for by choosing to rely on development release. Robie signature.asc Description: PGP signature -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
On Mon, Jun 17, 2024 at 11:20:48AM GMT, Robie Basak wrote:
> Thank you for working on this! It looks like it will be useful to have
> that metadata there.
>
> On Sat, Jun 15, 2024 at 12:46:15AM +0200, Benjamin Drung wrote:
> > Packaging Metadata:
> > {"type":"deb","os":"ubuntu","osVersion":"24.10","name":"chaos-marmosets","version":"0.2.0-1","architecture":"amd64"}
>
> Technically, we don't know it's going to be called 24.10 yet. For
> example, 6.04 became 6.06. For this reason, I believe nothing in our
> infrastructure makes this assumption prior to release and we shouldn't
> start now. I suggest not locking things down unnecessarily, and
> therefore "oracular" would be more suitable.
It is communicated directly at archive opening, when we upload the
base-files containing the os-release saying that:
$ grep 24.10 /etc/os-release
VERSION_ID="24.10"
VERSION="24.10 (Oracular Oriole)"
I believe it's fair to say at this point that even if we did
move a release, we'd not change the version anymore because
there will have been extensive communication over a multitude
of channels already using the version; third-party vendors
will have done testing against that specific version
string, etc.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
signature.asc
Description: PGP signature
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
Re: dpkg 1.22.6ubuntu11 enables ELF packaging metadata
Thank you for working on this! It looks like it will be useful to have
that metadata there.
On Sat, Jun 15, 2024 at 12:46:15AM +0200, Benjamin Drung wrote:
> Packaging Metadata:
> {"type":"deb","os":"ubuntu","osVersion":"24.10","name":"chaos-marmosets","version":"0.2.0-1","architecture":"amd64"}
Technically, we don't know it's going to be called 24.10 yet. For
example, 6.04 became 6.06. For this reason, I believe nothing in our
infrastructure makes this assumption prior to release and we shouldn't
start now. I suggest not locking things down unnecessarily, and
therefore "oracular" would be more suitable.
One might say that we've been following the pattern consistently for
long enough now that this is highly unlikely, which is true, but equally
there's no reason to lock it down here just for this IMHO. If in the
future we change the schedule, we won't have to rebuild things or leave
an inconsistency then. I also agree that user-facing stuff should avoid
using codenames that are hard to understand, but I don't think this is
one of those cases.
Robie
signature.asc
Description: PGP signature
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
dpkg 1.22.6ubuntu11 enables ELF packaging metadata
Hi everyone,
I just uploaded dpkg 1.22.6ubuntu11 to Ubuntu oracular. This version
enables ELF packaging metadata via dpkg-buildflags by default. ELF
objects will record the spec https://systemd.io/ELF_PACKAGE_METADATA/
Example data:
```
$ readelf --notes/usr/bin/leak-memory | grep Metadata
Packaging Metadata:
{"type":"deb","os":"ubuntu","osVersion":"24.10","name":"chaos-marmosets","version":"0.2.0-1","architecture":"amd64"}
```
This features is enabled by default. To opt out set:
DEB_BUILD_MAINT_OPTIONS = qa=-elfpackagemetadata
Thanks to Luca Boccassi for implementing it.
--
Benjamin Drung
Debian & Ubuntu Developer
--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
