Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Apr 19, 2021 at 06:31:48PM +0200, Julien Cristau wrote: > Can we defer these until after the AllowReleaseInfoChange change is > out, please? Done. I uploaded 1.8.2.3 with that one change as requested, and created a pu bug for it. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Apr 19, 2021 at 06:28:05PM +0200, Julian Andres Klode wrote: > On Mon, Apr 19, 2021 at 06:08:23PM +0200, Julien Cristau wrote: > > On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote: > > > I see. Nobody pinged me since then, but it is indeed fixed in the > > > 1.8.5 stable update that at least one release team member was > > > not exited about. > > > > > > https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5 > > > > > > So it's up to the release team if they want 1.8.5 or whether we'll have > > > to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that > > > is clear - I don't want to maintain a 1.8.2.z branch, it's more work > > > compared > > > to just following the normal stable apt updates, and there's a lot less > > > testing going on. > > > > > Please upload just that fix to buster; I don't care too much about the > > version number you pick. > > Is there a buster point release before bullseye release, or should that > be in buster-updates? > I don't know. It needs to make its way to spu soon either way. > Given that buster is going to security support only soon anyway, I don't > mind where I apply security updates to that much :D > > > But I really do want to cherry-pick at least two other code fixes, and one > test > suite fix: > Can we defer these until after the AllowReleaseInfoChange change is out, please? Thanks, Julien > * > https://salsa.debian.org/apt-team/apt/-/commit/cfee71c6f2d1478ce4d4ed74ef690ae1350ea403 > > https://salsa.debian.org/apt-team/apt/-/commit/75f452c7309d66548c86a6526cbd65fc51a70039 > > (really just one change, but it was split over two releases, first > turned error to warning, next one ignores it completely; because it > was in 2 releases in main so I kept it separate :D) > > too, they'll make immediate configuration errors non-fatal. Currently > they are fatal in the sense that they are ignored and the upgrade runs > and then it just exits with 1 at the end. So it does not change the > outcome, it just makes the error reporting less silly. > > It is very likely that some upgrades on multi-arch systems fail erronously > without them. To be fair, the multi-arch aspect is also fixed by > > https://salsa.debian.org/apt-team/apt/-/commit/7f65fa3843abc476cbba65c808abc5dd77835815 > but that changes ordering results, and is not strictly necessary. > > * And I want > > https://salsa.debian.org/apt-team/apt/-/commit/cfc6870e9b8ad119219ce5dc1871531006bb2d71 > > to avoid people getting packages removed that stuff still depends on > because their prerm script failed. This might happen during an upgrade > to bullseye, and it's very very likely APT will not be able to recover from > it > - I've never successfully recovered a distribution upgrade that had a > failure in the > middle (and fwiw, all of them had, but they were my faults, really). > > * Also the testsuite-only change in > > https://salsa.debian.org/apt-team/apt/-/commit/730c5c861c32c9385dc862af8673984b12902343 > which makes things work reliably on debci armhf (no regression > potential, wheee) > > -- > debian developer - deb.li/jak | jak-linux.org - free software dev > ubuntu core developer i speak de, en >
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Apr 19, 2021 at 06:08:23PM +0200, Julien Cristau wrote: > On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote: > > I see. Nobody pinged me since then, but it is indeed fixed in the > > 1.8.5 stable update that at least one release team member was > > not exited about. > > > > https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5 > > > > So it's up to the release team if they want 1.8.5 or whether we'll have > > to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that > > is clear - I don't want to maintain a 1.8.2.z branch, it's more work > > compared > > to just following the normal stable apt updates, and there's a lot less > > testing going on. > > > Please upload just that fix to buster; I don't care too much about the > version number you pick. Is there a buster point release before bullseye release, or should that be in buster-updates? Given that buster is going to security support only soon anyway, I don't mind where I apply security updates to that much :D But I really do want to cherry-pick at least two other code fixes, and one test suite fix: * https://salsa.debian.org/apt-team/apt/-/commit/cfee71c6f2d1478ce4d4ed74ef690ae1350ea403 https://salsa.debian.org/apt-team/apt/-/commit/75f452c7309d66548c86a6526cbd65fc51a70039 (really just one change, but it was split over two releases, first turned error to warning, next one ignores it completely; because it was in 2 releases in main so I kept it separate :D) too, they'll make immediate configuration errors non-fatal. Currently they are fatal in the sense that they are ignored and the upgrade runs and then it just exits with 1 at the end. So it does not change the outcome, it just makes the error reporting less silly. It is very likely that some upgrades on multi-arch systems fail erronously without them. To be fair, the multi-arch aspect is also fixed by https://salsa.debian.org/apt-team/apt/-/commit/7f65fa3843abc476cbba65c808abc5dd77835815 but that changes ordering results, and is not strictly necessary. * And I want https://salsa.debian.org/apt-team/apt/-/commit/cfc6870e9b8ad119219ce5dc1871531006bb2d71 to avoid people getting packages removed that stuff still depends on because their prerm script failed. This might happen during an upgrade to bullseye, and it's very very likely APT will not be able to recover from it - I've never successfully recovered a distribution upgrade that had a failure in the middle (and fwiw, all of them had, but they were my faults, really). * Also the testsuite-only change in https://salsa.debian.org/apt-team/apt/-/commit/730c5c861c32c9385dc862af8673984b12902343 which makes things work reliably on debci armhf (no regression potential, wheee) -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Apr 19, 2021 at 06:01:18PM +0200, Julian Andres Klode wrote: > I see. Nobody pinged me since then, but it is indeed fixed in the > 1.8.5 stable update that at least one release team member was > not exited about. > > https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5 > > So it's up to the release team if they want 1.8.5 or whether we'll have > to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that > is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared > to just following the normal stable apt updates, and there's a lot less > testing going on. > Please upload just that fix to buster; I don't care too much about the version number you pick. Thanks, Julien
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Apr 19, 2021 at 05:40:09PM +0200, Christoph Berg wrote: > Re: Julian Andres Klode > > > We're getting close to the release of bullseye and it has been brought > > > to my attention that this bug is still unfixed in buster. Once we > > > release bullseye, this bug is going to run havoc for our buster users. > > > > That's not accurate. This is _only_ a problem for users of testing, > > where the codename changes from time to time. > > This *is* a problem for users of "buster" where the suite will be > changing from "stable" to "oldstable". (Yes, we do release buster > twice, once as stable and then as oldstable.) > > Unless the fix that has closed #931566 is also applied to the apt > version in buster, things will explode horribly. Oh I see. That's tricky. > > Changed-By: Julian Andres Klode > Changes: > apt (2.1.10) unstable; urgency=medium > . >* Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: > #931566) > > Notably, that needs to happen well before the bullseye release or else > systems will not be able to "apt-get update" non-interactively to > actually see the updated package. > > > For stable users, this is not a problem at all, more the opposite. Those > > poor souls who have stable in their sources.list won't suddenly get > > upgraded to bullseye. > > Yes, this part of the change is the good one. Pinning suite for > "buster" users is not. > > > > Can we somehow come up with a plan on how to handle this? Can we have a > > > fix in the next point release? Are there faster options than just > > > waiting some time after the next point release before we can release > > > bullseye, e.g. could the SRM allow an update to stable for the change of > > > an apt default to have the change earlier than the next point release? > > > > I have no intention of issuing a stable update. > > On 2020-08-10 you said: > > 17:04 juliank: is #931566 going to be fixed in buster as well? > 17:04 -zwiebelbot- Debian#931566: Don't complain about suite changes > (Acquire::AllowReleaseInfoChange::Suite should be "true") - > https://bugs.debian.org/931566 > 17:04 Myon: yes > 17:04 cool > 17:04 thanks I see. Nobody pinged me since then, but it is indeed fixed in the 1.8.5 stable update that at least one release team member was not exited about. https://salsa.debian.org/apt-team/apt/-/compare/1.8.2.2...1.8.5 So it's up to the release team if they want 1.8.5 or whether we'll have to cherry-pick a subset of it into a 1.8.2.3. I think my opinion on that is clear - I don't want to maintain a 1.8.2.z branch, it's more work compared to just following the normal stable apt updates, and there's a lot less testing going on. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en signature.asc Description: PGP signature
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Re: Julian Andres Klode > > We're getting close to the release of bullseye and it has been brought > > to my attention that this bug is still unfixed in buster. Once we > > release bullseye, this bug is going to run havoc for our buster users. > > That's not accurate. This is _only_ a problem for users of testing, > where the codename changes from time to time. This *is* a problem for users of "buster" where the suite will be changing from "stable" to "oldstable". (Yes, we do release buster twice, once as stable and then as oldstable.) Unless the fix that has closed #931566 is also applied to the apt version in buster, things will explode horribly. Changed-By: Julian Andres Klode Changes: apt (2.1.10) unstable; urgency=medium . * Default Acquire::AllowReleaseInfoChange::Suite to "true" (Closes: #931566) Notably, that needs to happen well before the bullseye release or else systems will not be able to "apt-get update" non-interactively to actually see the updated package. > For stable users, this is not a problem at all, more the opposite. Those > poor souls who have stable in their sources.list won't suddenly get > upgraded to bullseye. Yes, this part of the change is the good one. Pinning suite for "buster" users is not. > > Can we somehow come up with a plan on how to handle this? Can we have a > > fix in the next point release? Are there faster options than just > > waiting some time after the next point release before we can release > > bullseye, e.g. could the SRM allow an update to stable for the change of > > an apt default to have the change earlier than the next point release? > > I have no intention of issuing a stable update. On 2020-08-10 you said: 17:04 juliank: is #931566 going to be fixed in buster as well? 17:04 -zwiebelbot- Debian#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true") - https://bugs.debian.org/931566 17:04 Myon: yes 17:04 cool 17:04 thanks Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Sun, Apr 18, 2021 at 10:20:19PM +0200, Paul Gevers wrote: > Hi Julian, David, SRM, > > On Mon, 8 Jul 2019 09:11:48 +0200 Christoph Berg wrote: > > Fwiw, I think this needs fixing in buster, not just in unstable. > > We're getting close to the release of bullseye and it has been brought > to my attention that this bug is still unfixed in buster. Once we > release bullseye, this bug is going to run havoc for our buster users. That's not accurate. This is _only_ a problem for users of testing, where the codename changes from time to time. For stable users, this is not a problem at all, more the opposite. Those poor souls who have stable in their sources.list won't suddenly get upgraded to bullseye. > > Can we somehow come up with a plan on how to handle this? Can we have a > fix in the next point release? Are there faster options than just > waiting some time after the next point release before we can release > bullseye, e.g. could the SRM allow an update to stable for the change of > an apt default to have the change earlier than the next point release? I have no intention of issuing a stable update. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en signature.asc Description: PGP signature
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Hi Julian, David, SRM, On Mon, 8 Jul 2019 09:11:48 +0200 Christoph Berg wrote: > Fwiw, I think this needs fixing in buster, not just in unstable. We're getting close to the release of bullseye and it has been brought to my attention that this bug is still unfixed in buster. Once we release bullseye, this bug is going to run havoc for our buster users. Can we somehow come up with a plan on how to handle this? Can we have a fix in the next point release? Are there faster options than just waiting some time after the next point release before we can release bullseye, e.g. could the SRM allow an update to stable for the change of an apt default to have the change earlier than the next point release? Paul OpenPGP_signature Description: OpenPGP digital signature
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Re: Paul Wise 2019-09-02 <68737fa6fe19d3c2a71beadefbe69c661dd4d271.ca...@debian.org> > I recently had a similar issue with the backports repos: > > E: Repository 'https://incoming.debian.org/debian-buildd > buildd-stretch-backports-sloppy InRelease' changed its default priority for > apt_preferences(5) from 500 to 100. > E: Repository 'https://cdn-aws.deb.debian.org/debian stretch-backports-sloppy > InRelease' changed its default priority for apt_preferences(5) from 500 to > 100. > > This was because those suites got a couple of settings added to their > Release files in the apt repository: > > NotAutomatic: yes > ButAutomaticUpgrades: yes I think this is an example of something that this mechanism *should* catch, especially if the change would have gone the other way round. But the pain caused by testing->stable and oldstable->stable is still much more prevalent than the legitimate cases. If we don't find a solution that can tell these apart in a robust way, we should disable the whole thing in stable. Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Sun, 7 Jul 2019 20:39:10 +0200 Christoph Berg wrote: > I and #debian-devel in general were pretty surprised that our "buster" > systems couldn't properly "apt-get update" anymore today: I recently had a similar issue with the backports repos: E: Repository 'https://incoming.debian.org/debian-buildd buildd-stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100. E: Repository 'https://cdn-aws.deb.debian.org/debian stretch-backports-sloppy InRelease' changed its default priority for apt_preferences(5) from 500 to 100. This was because those suites got a couple of settings added to their Release files in the apt repository: NotAutomatic: yes ButAutomaticUpgrades: yes -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Fri, Jul 12, 2019 at 04:52:36PM +0200, Julian Andres Klode wrote:
> On Tue, Jul 09, 2019 at 03:31:07PM +0200, David Kalnischkies wrote:
> > On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote:
> > Anyway, that apt is enforcing the metadata isn't changing is a security
> > feature in that if you don't check an attacker can reply to a request to
> > foo-security with foo – perfectly signed and everything so apt has no
> > chance to detect this and the user believes everything is fine – even
> > seeing files downloaded from -security – while the user never receives
> > data for -security. foo has no Valid-Until header so the attacker can
> > keep that up for basically ever. Compared to that serving old versions
> > of -security itself is guarded by Valid-Until. Not serving any data has
> > basically the same result, but the errors involved might eventually
> > raise some alarm bells. So that is a good strategy to keep you from ever
> > installing security upgrades.
>
> This should not happen silently, as there'll be a warning that the
> name of the distro in the sources.list entry matches neither Codename
> nor Suite.
Codename and Suite match for Debian and -security:
http://deb.debian.org/debian/dists/buster/Release
http://security.debian.org/debian-security/dists/buster/updates/Release
They differ in Label (, Version) and Components only. As such I can
easily hand you the Release file for Debian if you ask for security.
Components has its own verify, but as both styles ("updates/main" and
"main") exist in realworld scenarios we can't be that strict about it.
Version is a fun one to verify as that would imply enforcing a specific
version scheme.
Yes, they have different keys, but while the option exists nobody really
configures signed-by per sources.list entry (or Release or …) for
various reasons. !33 wants to help with that, but will/does depend on
the metadata to match keys entries to sources.list entries.
bullseye has that "fixed" as Suite & Codename will vary for both
archives, but there weren't only positive replies about that and I am
not that arrogant to call all other repositories "broken" just because
they don't exactly copy the choices Debian made – and even if they would
they would be a candidate for the flipping.
I guess Julian was thinking of the Docker example I had in a later
message which is caught by having the wrong suite compared to the
sources.list; but that example I included mainly to show that Debians
interpretation of what Suite and Codename are might not be applicable to
other repositories – for Docker, it makes perfect sense to say that
"buster" is the suite they release stuff for. It is at least quite
obviously not the codename for their product.
> We also can't do a rollback attack from current testing to an older
> testing either, as the Date is not allowed to roll backwards.
Assuming I don't catch you on your first update I just have to pick
a stable release (update) after the last security update you have
to switch you over to never receiving security updates again (perhaps
staling -security in the bounds of Valid-Until until stable has caught
up with the next point release).
(Me being a MITM or an evil [https-]mirror operator of course)
> I'm not convinced we are adding any actual security here - we should
> just upgrade the warning for name mismatch between sources.list and
> Release file to an error for that.
So, to be clear, you think about reverting the entire thing for all
fields – which is considerably more than what this bugreport is asking
for.
It is a valid option of course, but personally I would like to hear
reasons to allowing Origin/Label to change – so far I have only seen
complains about (mostly) Suite and (some) Codename [and for the record,
I also got some positive replies for both, so regardless the default,
I would at the very least like to retain the option] – as that
jeopardises stuff like !33 as well.
> > A) For a user with "debian stable" in sources.list the change of the
> > Codename from buster to bullseye is a giant leap which should not
> > be done carelessly or even automatically in the background.
>
> That's true, but leaving the user stranded without updates is not
> helpful either.
I would personally consider no-upgrades a better scenario than
not-for-me-upgrades, but then I wouldn't classify a failing automatic
process as "stranding" given that human intervention is needed in both
scenarios.
> > B) For a user with "debian buster" in sources.list the change of the
> > Suite from e.g. stable to oldstable is an important event as well; not
> > right at this moment as there is a grace period, but security support is
> > about to end and plans should be set in motion on how to deal with that.
>
> It's not an important enough change to starve them from further security
> updates until they acknowledge it.
I said as much, perhaps a bit clearer a few lines below that, and I think
we have consent on that – it
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Tue, Jul 09, 2019 at 03:31:07PM +0200, David Kalnischkies wrote: > On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote: > > Anyway, this was not my thing so I might be misrepresenting or missing > > things :) > > First of all: I am happy that I still manage to "break the world" (FSVO > world) – in other words that is "my thing" , but I mostly forgot about > it as it was implemented two years ago at the start of the last > release cycle so that everyone would have enough time to cry at me … > worked brilliantly I guess… 😉 > > Anyway, that apt is enforcing the metadata isn't changing is a security > feature in that if you don't check an attacker can reply to a request to > foo-security with foo – perfectly signed and everything so apt has no > chance to detect this and the user believes everything is fine – even > seeing files downloaded from -security – while the user never receives > data for -security. foo has no Valid-Until header so the attacker can > keep that up for basically ever. Compared to that serving old versions > of -security itself is guarded by Valid-Until. Not serving any data has > basically the same result, but the errors involved might eventually > raise some alarm bells. So that is a good strategy to keep you from ever > installing security upgrades. This should not happen silently, as there'll be a warning that the name of the distro in the sources.list entry matches neither Codename nor Suite. We also can't do a rollback attack from current testing to an older testing either, as the Date is not allowed to roll backwards. I'm not convinced we are adding any actual security here - we should just upgrade the warning for name mismatch between sources.list and Release file to an error for that. > So, after establishing that metadata shouldn't change all the time, lets > look at the individual metadata pieces. Th recent thread on (not) using > suite/codename/version to refer to a release is not only done in > documentation and sources.list, but also in configuration for example in > apt_preferences (which apt could check theoretically) and unattended- > upgrades (which apt can't realistically check). Many of these > configuration pieces for break silently and/or run guard automatic > processes. > > People are also not very consistent in that they refer to an release in > different ways depending on the situation: While the sources.list might > refer to buster, the config for unattended upgrades could easily refer > to packages from stable to install automatically. > > > A) For a user with "debian stable" in sources.list the change of the > Codename from buster to bullseye is a giant leap which should not > be done carelessly or even automatically in the background. That's true, but leaving the user stranded without updates is not helpful either. Regarding the automatic upgrades: It seems that unattended-upgrades only considers upgrades that match the system's codename, so it won't automatically upgrade you to a new release. > > B) For a user with "debian buster" in sources.list the change of the > Suite from e.g. stable to oldstable is an important event as well; not > right at this moment as there is a grace period, but security support is > about to end and plans should be set in motion on how to deal with that. It's not an important enough change to starve them from further security updates until they acknowledge it. > A Release file can declare that it will "soon" change its the value of > a field to foo and apt clients can notify users about this so > they can prepare (or, for that matter help testing if they feel like > it). In exchange, if the change happens and apt knows it announced it > before it can choose to accept the change without explicit user > confirmation & I would propose enabling that for the Suite field. > > > So, lets see how that might look for Debian buster if I "will have had" > deployed a time machine: > > On 2019-06-11, perhaps a bit earlier: > > $ cat buster/Release > Codename: buster > Future-Codename: bullseye > Suite: testing > Future-Suite: stable > Future-Version: 10.0 > Future-Release-Notes: https://example.org/preparing-for-buster It turns out the only reasonable point in time to do that is during release, that is, stable would always contain Future-Suite: oldstable (and similar for others). Two reasons: * It's super annoying having to do those changes for all releases at like freeze time * If your machine was offline in the time between adding Future and the new stable release, it will still be broken. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en signature.asc Description: PGP signature
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Re: David Kalnischkies 2019-07-10 <20190710100502.yoe2umfxyiuglaw2@crossbow> > http://deb.devuan.org/devuan/dists/stable/Release > http://deb.devuan.org/merged/dists/stable/Release > > The two release pockets differ by Suite only. Hi, this bug is not about fixing problems that devuan and the others might have, this is about not breaking proper Debian Buster installations. I appreciate the efforts put into sanely handling half-broken repositories, but I'd still like to avoid the looming shitstorm on the stable->oldstable switch in two years. Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Tue, Jul 09, 2019 at 06:52:58PM +0200, Julien Cristau wrote:
> I think overall what you're trying to do here (the whole "notify the
> user they're out of date" thing) does not belong in apt. IMO it belongs
> in higher level tools that are going to heavily depend on the use case
> and so there's not really a good generic answer you can come up with at
> the lowest (apt) level.
Well, all user-facing clients use libapt for their equivalent of "apt
update". So your one-off python script, apt, apt-get, aptitude,
synaptics, all software centers, unattended-upgrades, apt-file … they
all share the code – and the data which is downloaded by one of them for
them all.
It is the choice of the frontend performing the update to present
encountered problems and apt{,-get} chooses to push the messages to the
usual list at the end of the run as it is common for apt. It could just
as well do nothing or open a blicking popup dialog, whatever seems
sensible for both the user using the current frontend AND for the other
frontends – so yes, there needs to be a default generic good answer on
the lowest level as we otherwise need to tell users to pick ONE frontend
and use that for the rest of their life exclusively.
Earmarking potentially dangerous data in hope that all frontends will
implement proper safety procedures while dealing with them doesn't work.
That was quite (in)visible for unauthenticated packages and I am very
happy we got mostly right of it by now.
apt{,-get} could choose to implement checks if the changed metadata
breaks its configuration – and we might do if I am really bored – but
that would indeed be lots of code and we still have the problem that the
other frontends might be broken by the new data.
#931524 and #931649 alone show that user config is easily broken & that
users frequently miss obvious and well-known changes to repositories. Or
did we already forget the recent outcry as oldoldstable left the
mirrors? And that is the main archive of your distribution. I doubt
users are following changes in 3rdparty repos they are using any closer.
I agree that apt{,-get} isn't the best place to keep you posted about
this stuff, but libapt is the best (because it is the only) place to
download the data all frontends can use and if present apt{,-get} should
make use of it as users expect it to help them. Even better if there are
higher level tools making better use of the data!
BUT that isn't about notifying users or breaking configuration even if
that is the most visible in practice. You don't have to look particular
far for an opportunity to create disaster (at least in the eyes of the
user) if you allow an attacker to change one metadata field:
http://deb.devuan.org/devuan/dists/stable/Release
http://deb.devuan.org/merged/dists/stable/Release
The two release pockets differ by Suite only. [Okay, they differ by
Label, too, so apt would catch that – but that looks more like an
accident than deliberate. And how much impact can a Label change have…
that should be automatic, too! btw "Debian stable" and "Debian
stable-security" differ by Label only in buster].
(not an endorsement btw, just a semi-random pick from the census)
Or because everyone loves docker lets look there:
https://download.docker.com/linux/debian/dists/buster/Release
https://download.docker.com/linux/debian/dists/stretch/Release
https://download.docker.com/linux/raspbian/dists/buster/Release
https://download.docker.com/linux/ubuntu/dists/artful/Release
https://download.docker.com/linux/ubuntu/dists/zesty/Release
These indeed differ by Suite only and are therefore freely exchangeable
if we allow unsanctioned Suite changes.
Having Oracle in your trusted keyring? Great! Then I will serve you this
the next time you request the Debian unstable repository:
https://oss.oracle.com/debian/dists/unstable/main/binary-i386/Release
[caught by codename – or more realistically by signed-by, expect nobody
uses that. MR !33 to the rescue but this requires correct and relatively
stable metadata as well]
The last one is just the literally first hit on duckduckgo for '"Origin:
Debian" Release' for me. It isn't particularity hard to find others with
better usability factors.
So if the proposed solution is over engineered I am all ears for
alternatives which deal with these issues.
Best regards
David Kalnischkies
P.S.: Pinning and other actions in libapt by codename was the first big
feature I implemented a decade ago. So in Debian terms it isn't that old
and indeed lots of documentation still uses suite names for this – and
in many cases that isn't wrong. Other frontends got that feature even
later – assuming it is implemented at all by now. Reading mails worded
as if codenames were a universal truth is quite funny in that context.
signature.asc
Description: PGP signature
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Tue, Jul 9, 2019 at 15:55:02 +0200, Christoph Berg wrote: > Re: David Kalnischkies 2019-07-09 <20190709133107.gvib2ibj6w36j447@crossbow> > > A Release file can declare that it will "soon" change its the value of > > a field to foo and apt clients can notify users about this so > > Fwiw, I think this is seriously overengineered. We can't really test > it, and there's high chances the feature will break (itself or > something else) more than it might fix in practise. > Definitely agreed. I think overall what you're trying to do here (the whole "notify the user they're out of date" thing) does not belong in apt. IMO it belongs in higher level tools that are going to heavily depend on the use case and so there's not really a good generic answer you can come up with at the lowest (apt) level. Cheers, Julien
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Re: David Kalnischkies 2019-07-09 <20190709133107.gvib2ibj6w36j447@crossbow> > A) For a user with "debian stable" in sources.list the change of the > Codename from buster to bullseye is a giant leap which should not > be done carelessly or even automatically in the background. I agree that stopping "apt-get update" here makes sense. > B) For a user with "debian buster" in sources.list the change of the > Suite from e.g. stable to oldstable is an important event as well; not > right at this moment as there is a grace period, but security support is > about to end and plans should be set in motion on how to deal with that. Refusing to continue with non-interactive (!) operation here is totally wrong. The system is configured perfectly fine to use "buster", and just because buster is moving from testing to stable, or stable to oldstable doesn't justify breaking 100s of installations in my data center. The message that I should probably be looking into dist-upgrading my data center will be sent to me in form of debian-announce, the general news, or whatever other mechanism. I need to get that message once, not once per system. What I don't need is 100s of systems breaking "apt-get update" because that means I won't get monitoring messages about security updates either (where one message per system makes sense). > APT isn't a newspaper, but at least for me it seems wrong that the one > tool people associate with getting packages has absolutely nothing to > say about big events at the vendor who issues my packages. If you chose to send a message, that's ok, but the message must not break non-interactive, unattended apt-get operation. (At the very least, don't break "apt-get". For more fancy things, there's "apt".) > A Release file can declare that it will "soon" change its the value of > a field to foo and apt clients can notify users about this so Fwiw, I think this is seriously overengineered. We can't really test it, and there's high chances the feature will break (itself or something else) more than it might fix in practise. Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Jul 08, 2019 at 12:19:36PM +0200, Julian Andres Klode wrote: > Anyway, this was not my thing so I might be misrepresenting or missing > things :) First of all: I am happy that I still manage to "break the world" (FSVO world) – in other words that is "my thing" , but I mostly forgot about it as it was implemented two years ago at the start of the last release cycle so that everyone would have enough time to cry at me … worked brilliantly I guess… 😉 Anyway, that apt is enforcing the metadata isn't changing is a security feature in that if you don't check an attacker can reply to a request to foo-security with foo – perfectly signed and everything so apt has no chance to detect this and the user believes everything is fine – even seeing files downloaded from -security – while the user never receives data for -security. foo has no Valid-Until header so the attacker can keep that up for basically ever. Compared to that serving old versions of -security itself is guarded by Valid-Until. Not serving any data has basically the same result, but the errors involved might eventually raise some alarm bells. So that is a good strategy to keep you from ever installing security upgrades. So, after establishing that metadata shouldn't change all the time, lets look at the individual metadata pieces. Th recent thread on (not) using suite/codename/version to refer to a release is not only done in documentation and sources.list, but also in configuration for example in apt_preferences (which apt could check theoretically) and unattended- upgrades (which apt can't realistically check). Many of these configuration pieces for break silently and/or run guard automatic processes. People are also not very consistent in that they refer to an release in different ways depending on the situation: While the sources.list might refer to buster, the config for unattended upgrades could easily refer to packages from stable to install automatically. A) For a user with "debian stable" in sources.list the change of the Codename from buster to bullseye is a giant leap which should not be done carelessly or even automatically in the background. B) For a user with "debian buster" in sources.list the change of the Suite from e.g. stable to oldstable is an important event as well; not right at this moment as there is a grace period, but security support is about to end and plans should be set in motion on how to deal with that. (A & B assume Debian for brevity. In other repos changes can have even bigger/different effects, but that mail would get even longer if I would write yet more examples) Both situations deserve nagging the user in my opinion – preferably with a pointer to specific details [which apt clients do if a Release-Notes field is present in the Release file] – even if we ignore the risk of breaking existing configuration on the system. APT isn't a newspaper, but at least for me it seems wrong that the one tool people associate with getting packages has absolutely nothing to say about big events at the vendor who issues my packages. That said, while I might be able to hold my ground on A) with the remark that setting Acquire::AllowReleaseInfoChange::Codename "true" on system where you know that upgrade processes can be ignored. (setting it to true by default and letting d-i disable it for >= standard feels dirty). On B) I got serious backlash through and I might agree given that I said "grace period" myself and an apt client is basically removing that period now. Not really what I wanted… As such I would propose what Julian has hinted at already as we talked about it on IRC yesterday: A Release file can declare that it will "soon" change its the value of a field to foo and apt clients can notify users about this so they can prepare (or, for that matter help testing if they feel like it). In exchange, if the change happens and apt knows it announced it before it can choose to accept the change without explicit user confirmation & I would propose enabling that for the Suite field. So, lets see how that might look for Debian buster if I "will have had" deployed a time machine: On 2019-06-11, perhaps a bit earlier: $ cat buster/Release Codename: buster Future-Codename: bullseye Suite: testing Future-Suite: stable Future-Version: 10.0 Future-Release-Notes: https://example.org/preparing-for-buster $ apt update N: Repository '…' changes its 'Suite' value from 'testing' to 'stable' soon. N: Repository '…' changes its 'Codename' value from 'buster' to 'bullseye' soon. N: Repository '…' changes its 'Version' value from '' to '10.0' soon. N: More information about this can be found online in the Release notes at: https://example.org/preparing-for-buster Some of these N: will be wrong depending on how you choose to refer to the repository in your sources.list – I guess we could filter with some heuristics – or not, I kinda like how that lists the options. Anyway, as N are notices they do not change the exit status o
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Jul 08, 2019 at 11:50:49AM +0200, Julien Cristau wrote: > On Mon, Jul 8, 2019 at 11:41:55 +0200, Julian Andres Klode wrote: > > > On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote: > > > Control: severity -1 serious > > > > > > On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote: > > > > Control: tags -1 buster sid > > > > > > > > Re: To Debian Bug Tracking System 2019-07-07 > > > > <20190707183910.ga22...@msg.df7cb.de> > > > > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; > > > > > > > > Fwiw, I think this needs fixing in buster, not just in unstable. > > > > > > > Agree. The user experience from this is horrible. > > > > We'll figure out what to do. That said, let's not rush things, > > we've got until the next release to figure out what to do and > > fix it (a fix does not help anyone affected by the change right > > now anyway). > > > > We likely do not want to allow arbitrary changes of Suite. One > > option is to introduce a new field that specifies that the Suite > > field will change to something else soon, and a field to indicate > > release notes for that change. > > > > These are important security and safety features we don't just want > > to kill because they are inconvenient. (The big reason this was > > introduced is that if you allow this to change, you can just serve > > someone an "old" oldstable repository (that still says stable) instead > > of stable, and thus starve the clients from updates). > > > How does the current behaviour fix that? If you replay an old repo the > user is still starved from updates (how could they not be, anyway). But > in addition now users using a legitimate repo are *also* starved from > updates, so everyone loses? Well, I guess I was off a bit, you replay a current oldstable - you can't roll back to older Date values (also, Valid-Until). Without the AllowReleaseInfo changes, all the user woudl get is a warning that there's a mismatch, but no breakage and non-interactive systems would thus silently not get updates. Now they'll have a failing systemd timer, which I guess is better. Sure, yes, in practice this means Codename changed as well. I'm not sure there really are many repositories where only Suite can change in a way that's security-relevant. Though, in any case, you might still end up with broken pinning, thus it seems like a good idea to try to prevent this as long as we don't mess up things. Anyway, this was not my thing so I might be misrepresenting or missing things :) -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Jul 8, 2019 at 11:41:55 +0200, Julian Andres Klode wrote: > On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote: > > Control: severity -1 serious > > > > On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote: > > > Control: tags -1 buster sid > > > > > > Re: To Debian Bug Tracking System 2019-07-07 > > > <20190707183910.ga22...@msg.df7cb.de> > > > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; > > > > > > Fwiw, I think this needs fixing in buster, not just in unstable. > > > > > Agree. The user experience from this is horrible. > > We'll figure out what to do. That said, let's not rush things, > we've got until the next release to figure out what to do and > fix it (a fix does not help anyone affected by the change right > now anyway). > > We likely do not want to allow arbitrary changes of Suite. One > option is to introduce a new field that specifies that the Suite > field will change to something else soon, and a field to indicate > release notes for that change. > > These are important security and safety features we don't just want > to kill because they are inconvenient. (The big reason this was > introduced is that if you allow this to change, you can just serve > someone an "old" oldstable repository (that still says stable) instead > of stable, and thus starve the clients from updates). > How does the current behaviour fix that? If you replay an old repo the user is still starved from updates (how could they not be, anyway). But in addition now users using a legitimate repo are *also* starved from updates, so everyone loses? Cheers, Julien
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Re: Julian Andres Klode 2019-07-08 <20190708113658.ga10...@debian.org> > We likely do not want to allow arbitrary changes of Suite. One > option is to introduce a new field that specifies that the Suite > field will change to something else soon, and a field to indicate > release notes for that change. If you plan to add anything that relies on "notifications" in the Release file, please keep in mind that the mechanism still needs to work in setups that are supposed to run unattended. Build chroots in my case, and probably most machines in any data center where there is no admin typing "apt(-get) update" every other day. Please don't over-engineer it. Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Jul 08, 2019 at 10:17:27AM +0200, Julien Cristau wrote: > Control: severity -1 serious > > On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote: > > Control: tags -1 buster sid > > > > Re: To Debian Bug Tracking System 2019-07-07 > > <20190707183910.ga22...@msg.df7cb.de> > > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; > > > > Fwiw, I think this needs fixing in buster, not just in unstable. > > > Agree. The user experience from this is horrible. We'll figure out what to do. That said, let's not rush things, we've got until the next release to figure out what to do and fix it (a fix does not help anyone affected by the change right now anyway). We likely do not want to allow arbitrary changes of Suite. One option is to introduce a new field that specifies that the Suite field will change to something else soon, and a field to indicate release notes for that change. These are important security and safety features we don't just want to kill because they are inconvenient. (The big reason this was introduced is that if you allow this to change, you can just serve someone an "old" oldstable repository (that still says stable) instead of stable, and thus starve the clients from updates). -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
On Mon, Jul 08, 2019 at 11:19:28AM +0200, Andreas Beckmann wrote: > Followup-For: Bug #931566 > Control: tag -1 experimental > > IMHO, the intention behind this change is good. > The implementation was completely untested. Well, it's kind of hard to > properly test such a change. > The documentation is not really helpful. > > I initially thought this was caused by my rather complicated setup. > Until I experienced it on a machine installed with buster two weeks ago > and not yet "customized" by me. > > Multiple errors like this: > E: Repository 'http://security-cdn.debian.org stretch/updates InRelease' > changed its 'Suite' value from 'stable' to 'oldstable' > N: This must be accepted explicitly before updates for this repository > can be applied. See apt-secure(8) manpage for details. > > apt-secure(8) says: > > INFORMATION CHANGES >A Release file contains beside the checksums for the files in the >repository also general information about the repository like the >origin, codename or version number of the release. > >This information is shown in various places so a repository owner >should always ensure correctness. Further more user configuration >like apt_preferences(5) can depend and make use of this >information. >Since version 1.5 the user must therefore explicitly confirm >changes to signal that the user is sufficiently prepared e.g. for >the new major release of the distribution shipped in the >repository (as e.g. indicated by the codename). > > OK, and what do I do now? How do I confirm that? > > It took me some time to discover --allow-releaseinfo-change in > apt-get(8). Please let's not discuss the manpage here. Bug#879786 has been filed for that back in Oct 2017. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Followup-For: Bug #931566 Control: tag -1 experimental IMHO, the intention behind this change is good. The implementation was completely untested. Well, it's kind of hard to properly test such a change. The documentation is not really helpful. I initially thought this was caused by my rather complicated setup. Until I experienced it on a machine installed with buster two weeks ago and not yet "customized" by me. Multiple errors like this: E: Repository 'http://security-cdn.debian.org stretch/updates InRelease' changed its 'Suite' value from 'stable' to 'oldstable' N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details. apt-secure(8) says: INFORMATION CHANGES A Release file contains beside the checksums for the files in the repository also general information about the repository like the origin, codename or version number of the release. This information is shown in various places so a repository owner should always ensure correctness. Further more user configuration like apt_preferences(5) can depend and make use of this information. Since version 1.5 the user must therefore explicitly confirm changes to signal that the user is sufficiently prepared e.g. for the new major release of the distribution shipped in the repository (as e.g. indicated by the codename). OK, and what do I do now? How do I confirm that? It took me some time to discover --allow-releaseinfo-change in apt-get(8). Andreas PS: my apt pinning is exactly setup to prevent unwanted release changes even if sources.list contains everything from wheezy to experimental ;-)
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Control: severity -1 serious On Mon, Jul 08, 2019 at 09:11:48AM +0200, Christoph Berg wrote: > Control: tags -1 buster sid > > Re: To Debian Bug Tracking System 2019-07-07 > <20190707183910.ga22...@msg.df7cb.de> > > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; > > Fwiw, I think this needs fixing in buster, not just in unstable. > Agree. The user experience from this is horrible. Cheers, Julien
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Control: tags -1 buster sid Re: To Debian Bug Tracking System 2019-07-07 <20190707183910.ga22...@msg.df7cb.de> > Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; Fwiw, I think this needs fixing in buster, not just in unstable. Christoph
Bug#931566: Don't complain about suite changes (Acquire::AllowReleaseInfoChange::Suite should be "true")
Package: apt Version: 1.8.1 Severity: important Hi, I and #debian-devel in general were pretty surprised that our "buster" systems couldn't properly "apt-get update" anymore today: + apt-get update Get:1 http://mirror.hetzner.de/debian/packages buster InRelease [118 kB] Get:2 http://security.debian.org/debian-security buster/updates InRelease [39.1 kB] Get:3 http://ftp.de.debian.org/debian buster InRelease [118 kB] Hit:4 http://apt.postgresql.org/pub/repos/apt buster-pgdg InRelease Hit:5 http://atalia.postgresql.org/pub/repos/apt buster-pgdg-testing InRelease Reading package lists... E: Repository 'http://mirror.hetzner.de/debian/packages buster InRelease' changed its 'Suite' value from 'testing' to 'stable' E: Repository 'http://security.debian.org/debian-security buster/updates InRelease' changed its 'Suite' value from 'testing' to 'stable' E: Repository 'http://ftp.de.debian.org/debian buster InRelease' changed its 'Suite' value from 'testing' to 'stable' For me, this killed the update process for the build chroots for apt.postgresql.org. I guess the same thing will happen once buster is turning oldstable, breaking out all our stable installations out there. IMHO that's bad. It might make sense to send a signal to people tracking suites, but if sources.list is on a codename, it doesn't make any sense to break systems that are totally fine. Please consider setting Acquire::AllowReleaseInfoChange::Suite "true"; Or maybe silently do the change when running non-interactively. Thanks. Christoph PS: Please document AllowReleaseInfoChange.
