Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks
Hi! On Wed, 2024-01-03 at 15:04:01 -0700, Sam Hartman wrote: > > "Guillem" == Guillem Jover writes: > Guillem> At least the dpkg behavior seems entirely > Guillem> correct to me and required for safe upgrades ( > > Can you help me understand the sentence above? > Where is the case where this behavior is needed for safe upgrades? > (I am asking out of curiosity; I'm guessing it's some corner case with > essential packages, but I would like to understand.) I should probably have qualified that statement, where safe upgrades is restricted to the specific scenarios at hand. In any case, I don't think this is specific to essential packages, this seems more general. But of course the effects of not supporting such safe upgrades for essential packages would be potentially more severe (although an essential package can never be a conflictor as dpkg would refuse to remove it). In essence this involves a cyclic restriction where a set of packages are stating they cannot be unpacked at the same time, but later versions might be. There are several subcases for this depending on the strength of the dependencies from each side, whether these are versioned, and also whether the package manager front-end or the user decides whether to fully evict a conflictor or wants to upgrade to a latest version that can co-exist. What the current behavior permits is to safely intertwine an unpack, a conflictors files transfer and its removal in the same step, so that the front-end or the user does not need to apply --force-* options to forcibly remove the conflictor while breaking the dependency system for an undetermined amount of time, to be able to proceed with such upgrade. I don't see any other way around this kind of upgrade that does not break the dependency system besides the current behavior (if someone does, I'm happy to hear it). Of course there's always the option to prohibit those kinds of relationships, which means the behavior never is brought up, and there's no need either to be concerned about whether it needs to be supported or not. But that seems overly restrictive, because once or if you need to express that kind of relationship, then that path would be closed. All this being said, when Helmut brought this up, I noticed that dpkg is expecting to be hand held by giving to it the proper order for these operations (as front-ends do), or it will either fail to upgrade depending on the scenario, or unnecessarily use the conflictor eviction behavior, which are not ideal (but do not break the dependency system). I've got some test cases (Conflicts vs Breaks, Conflicts vs Conflicts), which I should expand to add addition restrictions in the form of Depends and Pre-Depends, unversioned relationships and similar, after which I'll look into improving the way dpkg schedules its processing queue in these cases so that it is a bit smarter about them and does not fail so easily or uses this behavior unnecessarily when there's a path forward at hand. But this seems besides the point of the conflictor eviction behavior. Thanks, Guillem
Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks
> "Guillem" == Guillem Jover writes: Guillem> At least the dpkg behavior seems entirely Guillem> correct to me and required for safe upgrades ( Can you help me understand the sentence above? Where is the case where this behavior is needed for safe upgrades? (I am asking out of curiosity; I'm guessing it's some corner case with essential packages, but I would like to understand.) --Sam
Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks
Hi! On Fri, 2023-12-15 at 16:40:09 +, Sean Whitton wrote: > On Fri 01 Dec 2023 at 02:11pm +01, Helmut Grohne wrote: > > §7.4 currently starts with: > > > > When one binary package declares a conflict with another using a > > Conflicts field, dpkg will refuse to allow them to be unpacked on > > the system at the same time. > > > > I believe this is technically wrong. There are situations where dpkg > > will allow such unpacks to temporarily co-exist. §6.6 goes into further > > detail and is accurate. > > Thank you for the detailed report. > > Do the dpkg and apt people think that the bug here is just in Policy, or > are there any code changes under consideration in response to this work? I think it is just a documentation issue in the Debian Policy, yes. At least the dpkg behavior seems entirely correct to me and required for safe upgrades (and definitely not something like an accidental regression as it has behaved that way since pretty much the beginning of its git history). In addition I think the paragraph in §7.4 that states: ,--- A package will not cause a conflict merely because its configuration files are still installed; it must be at least “Half-Installed”. `--- could also be clarified that what is stated here does not apply either to conflicting packages that are being removed, as those will be in half-installed state. Perhaps as part of this, also make this state change explicit in §6.6.2.3. Thanks, Guillem
Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks
Hello, On Fri 01 Dec 2023 at 02:11pm +01, Helmut Grohne wrote: > §7.4 currently starts with: > > When one binary package declares a conflict with another using a > Conflicts field, dpkg will refuse to allow them to be unpacked on > the system at the same time. > > I believe this is technically wrong. There are situations where dpkg > will allow such unpacks to temporarily co-exist. §6.6 goes into further > detail and is accurate. Thank you for the detailed report. Do the dpkg and apt people think that the bug here is just in Policy, or are there any code changes under consideration in response to this work? -- Sean Whitton signature.asc Description: PGP signature
Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks
Package: debian-policy Version: 4.6.2.0 X-Debbugs-Cc: debian-d...@lists.debian.org, de...@lists.debian.org Hi, first of all huge thanks to David, Guillem and Julian for all of their explanations. In large parts, this bug report is yours and I'm just the one writing it down. §7.4 currently starts with: When one binary package declares a conflict with another using a Conflicts field, dpkg will refuse to allow them to be unpacked on the system at the same time. I believe this is technically wrong. There are situations where dpkg will allow such unpacks to temporarily co-exist. §6.6 goes into further detail and is accurate. Suppose we have two arch:all packages a version 1 and b version 1 both of which are installed. Now we attempt to install a version 2, which happens to declare "Conflicts: b (<< 2)". We may therefore mark b for removal echo "b:all deinstall" | dpkg --set-selections and proceed to installing a: dpkg --auto-deconfigure --unpack a_2.deb When we do this, dpkg will unpack a version 2 before removing the files of b version 1. I argue this is very briefly allowing these packages to be unpacked at the same time as the next thing dpkg does is removing b's files. This situation can be forced if we add package b version 2, which declares "Breaks: a (<< 2)" and attempt to install both. apt figures that it has to temporarily remove b and hence issues the selection above. Then it proceeds to unpacking both packages. The difference actually is rather subtle. As dpkg is tracking ownership of files, one should not be observing a difference. What one can see is that a.preinst version 2 is run at a time where b version 1 is still unpacked (and that's fine as the statement only talks about unpack). The effects of concurrent unpack are theoretically not observable, due to dpkg tracking files. However when you add aliasing to the mix, dpkg can now delete files that are still needed via differences in aliasing. That way - and I am fully aware that this violates fundamental assumptions of dpkg - we can make the order of unpacks visible and demonstrate that indeed a version 2 is unpacked before b version 1 has its files removed. All of this is fully in line with the long description in §6.6. What I take issue with is the executive summary at the start of §7.4. In case you like some kind of test case to tinker with, I'm attaching a script that demonstrates the situation. Helmut conflict-demo.sh Description: Bourne shell script
