Hi Simon, On Thu, Dec 08, 2022 at 05:20:20PM +0100, Simon Richter wrote: > I've also started work on getting usrmerge back into a sensible state, > current progress is at > > https://salsa.debian.org/sjr/dpkg/-/tree/wip-canonical-paths
Thank you. I've looked into this approach and I don't think I fully understand the intended semantics yet. Can you verify whether my understanding is correct and extend it where necessary? Binary packages may ship a new file in control.tar called "canonical". This file must contain an even number of lines. Each line must be an absolute path. Odd lines identify locations of symbolic links. Even lines identify the corresponding link target. It is unclear to me whether more than one package may ship such a canonical file. In case only one package may ship a canonical file, it is unclear to me whether a canonical file can be moved between packages. When dpkg encouners a canonial file, it will not rewrite its on-disk database structures, but load the canonical mapping into memory. dpkg will not move files around nor turn relevant directories into symbolic links. > I'd like to avoid a separate tool like dpkg-divert, because we need to be > able to set up aliases without being able to run maintainer scripts, so > these are useful from (c)debootstrap when setting up a foreign architecture. While this is a nice property, I do not think that it is required. The way this is implemented in debootstrap is adding extra scripts for setting up the symbolic links. And for mmdebstrap, you can choose between a hook achieving something similar and installing the usrmerge package, which performs the conversion at postinst time. I agree that this state of affairs is suboptimal, but at the same time it doesn't seem like the sky is falling. Why do you think that it must be possible to set up aliases without running maintainer scripts? It seems like I am missing some important aspect. > Ideally we'd be able to get rid of any special usrmerge handling in other > tools that way as well, and gain some infrastructure that will also be > useful for declarative diversions. Do I understand correctly that you'd like (some component of) dpkg to perform the moving of files and creation of symlinks? > The obvious interface breakage would be dpkg-query, which would probably > behave like > > $ dpkg-query -S /usr/bin/ls > coreutils: /bin/ls > > but we have all the relevant file names handy at this point so we can have a > different format as well. I think that dpkg -S is neither critical nor is there a way for it to not break. If you ask it for some file, it can either return the canonical location of a file or the actual location. Depending the use one or the other may be desired, so some uses do get broken in any case. You just choose its behaviour here. I think this is totally fine. > Transition would be a dpkg version that conflicts with usrmerge, and a > systemd package that declares the alias and pre-depends on a recent enough > version of dpkg, that will work for both upgrades and new installations. Why do you think that systemd needs to be entangled into this? I note that systemd is not essential. Therefore having systemd set up aliasing would not implement the CTTE decision, becaue an essential chroot would then be unmerged. I think this is fixable by not touching systemd. Instead, usrmerge can acquire the canonical table. If this method also does the conversion, the conversion code could disappear from the usrmerge package and it could be merged with usr-is-merged. Do you consider this adaption reasonable? Another possible candidate for the canonical table could be base-files. > If dpkg external to the new installation is used, the files will be moved > during unpacking when the systemd package is encountered, if the files are > first manually unpacked, and then overwritten later during a second round, > that should still work (but we need to be aware of this case). I think you are implicitly referrig to mmdebstrap here. Your proposal implements an important (to me) property: A patched dpkg will continue to be able to install packages on unmerged chroots. > The most annoying part will be identifying all the corner cases, like > crossgrades from ppc32 to ppc64, or from Debian to Devuan and back, and > writing test cases for those. As much as I would like to solve each and every problem, corner cases are a problem already. We noticed some of them when usrmerge became mandatory and things started breaking again. At the same time, the encountered corner cases were handled quite quickly. The same approach seems viable here to me. >From my point of view, the really important property is that upgrades that happen to involve file moves between locations and packages no longer accidentally overwrite or delete files. As far as I understand your proposal, such a situation is prevented by raising an error for conclicts between canonical and non-canonical locations. It can be avoided by adding relevant Breaks and Replaces then. Let me also compare your patch to my vapor ware. There is quite some similarity, so I'll focus on the differences instead. Rather than add a new option to dpkg for setting up aliases, you turn them into a canoical mapping shipped in a binary package. As a consequence, canonical mappings are a feature that administrators cannot use with ease, but it can be used by packages without invoking maintainer scripts unlike my proposal. An aspect that I left open as a question was whether the addition of aliases was something reversible. In uau's patch, this is irreversible as it modifies the dpkg database on the fly. In yours, it becomes reversible. Do you consider this is accurate? Can you also share your long term vision of this? Consider the time when this would be merged and we'd update all packages to no longer ship files in aliased directories. Then we wait a release. Should that canonical file go away at that point and just leave the symbolic links around or do we keep the canonical file forever? Helmut
