On Wed, Sep 06, 2023 at 11:27:02AM +0200, Guillem Jover wrote: > Hi! > > On Fri, 2023-09-01 at 08:43:55 +1200, Michael Hudson-Doyle wrote: > > Recently the topic of exploiting newer instructions without dropping > > support for older machines has come up several times inside Ubuntu > > engineering. I understand this topic has come up several times in the past > > for Debian as well, but nothing has really come of it to date. > > I also had a chat about this with Matthias Klose (CCed) around 2022-05. > > > I've spent a while thinking through the options and coming up with a design > > and wrote some notes into a wiki page: > > https://wiki.debian.org/ArchitectureVariants > > I think we are already doing 1, 2 and 3. I agree 4 is just wrong. And > something like 5 is what I suggested to Matthias for Ubuntu when we > last discussed it as the best way to go about this. > > I'm not sure I entirely agree with the requirements you set forth > though: > > - I think such optimized builds might need to be done with "special > toolchains" (these could simply be wrappers over the host compiler > passing the appropriate flags via command-line or via specs or > similar, not necessarily full blown toolchains), passing these via > something like dpkg-buildflags seems currently unreliable, as I don't > think we have full coverage in packages (neither for all compilers > available)? Although it would be better as it would centralize the > management. (For reference this is in part how rpm handles this: > https://github.com/rpm-software-management/rpm/blob/master/rpmrc.in) > - Perhaps that's a limitation from the archive software side, but > requiring to place the binary packages in the same pool seems > rather restrictive (it forces different filenames for example). > - I guess it might be nice for the ISA to be passed down to the > dpkg tools, but I don't think this is strictly necessary? A > frontend like apt could also decide based on metadata in say the > Release file, although not having the actual installed package > metadata on whether it was a different ISA build or not would make > its job more inconvenient. In any case I don't have a big issue > with recording this via dpkg-gencontrol or similar if necessary. > > On the specific implementation details: > > - Changing the Architecture format (as in adding colons there) seems > like a non-starter, and I expect that would break lots of things > (I mean it could be done but I'm not sure it's worth it for this). > Recording this mostly as a hint than anything else, via another > field (if necessary at all) I think would be best. > - As covered in previous discussions, dpkg could (but I don't think > it's necessary) check whether the .deb is runnable on the current > hw, but that's tricky as chrootless installs need to be taken > into account, etc. It should certainly not be part of dependency > resolution. > - I'm not fond of having to change the binary package name format > either for this (name_version_arch.deb) even if at least dpkg > itself does not care (but I know other tools do care), and > depending on the format I'd expect things to break (this goes > back to the shared pool concern). > - If dpkg-architecture needs to be aware of this, then this might need > to be auto-detectable from just the current toolchain being used. > > Some of the above problems could perhaps be avoided if we introduced > a concept of architecture aliases/ISAs (similar to what rpm has), which > would side-step the pool sharing issue, the binary package renaming, > etc. One big issue with this is that it requires for dpkg to have an > exhaustive table of all such aliases, and if there's ever a new alias > added, old dpkg versions need to be updated or they will not understand > what they match with. So this does not seem ideal either. So I guess this > is a variation over your proposal, but perhaps this could still be used > in specific contexts, say only at build-time (but not for dependency > relationships), for repo management (say binary-arm64v9/Packages.xz), > or binary package names where the field would specify the actual name > for the filename, say: > > Architecture: arm64 > ArchitectureIsa: arm64v9 > > or maybe better: > > Architecture: arm64 > ArchitectureIsa: v9 > > resulting in dpkg-deb generating: > > binpkg_1.0-1_arm64v9.deb > > but targeting arm64. I also think I prefer naming this explicitly as ISA > variants, if you will, than just architecture variants as that gives > way too much room (which perhaps we want, but then that has other > implications over compatibility), and for the field perhaps just Isa is > better, to avoid the implicit repetition of > ArchitectureInstructionSetArchitecture :), but that makes it less easy > to associate both as related.
I have thought more about this and I'm not particularly fond of the ArchitectureIsa name. While *this specific use case* is a variant of the architecture instruction set; you could just as well build other variants such as "compiled with -O3", "compiled with frame pointers", "compiled with -O0", or other shenanigans (I haven't thought about others outside compiler flags) Hence I prefer Architecture-Variant, Subarchitecture, or anything like that rather than have to invent another field or abuse this one the next time we want to build a special variant of an architecture with special optimizations for a special customer or whatnot. This has stalled a bit, but we really gotta get going on this, we should really have this working in APT by end of year or something. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en