Re: Apt feature request / suggestion

[Austin Moon]

Than you for the thoughtful reply.

Yes the apt upgrade submit ends there with no further error messages. When
moving that source list it proceeds and asks me if i want to install the
upgrades.

Please help me to understand the MITM scenario. So you're saying an
attacker could be able to redirect all sources except one? Is that even
technically likely? Has that ever happened?

I appreciate you explaining the risk, as I'm trying to understand why.
Please understand that this also prevents unattended upgrades from
proceeding, which aside from a real indication of MITM attack seems like
unnecessarily putting many systems at a security risk. (Also, shouldn't
https be used by default in the main sources list that includes security
repos? Doesn't https prevent MITM?)


Re space checking: I understand estimates are inaccurate. But it seems
better to fail on the side of caution in this instance. To be clear, I'm
not expecting to make waves for so many other users, this could be rolled
out with a flag option like (--check-space). I do think opt out should be
the default though.

I know there are separate partitions, but doesn't the estimation look at
say /boot when a kernel is queued for install? If it knows where to install
packages, it can check those exact locations for free space. (But still
imho better to fail by default)

I would take the conservative side of the estimate, as I've said the
uninstall process can be very complicated, especially when inside a VM with
no other space available. Usually means manually deleting files that upsets
dpkg and then having to reconfigure all that first before making apt happy
again.

I think 1.5GB is reasonable. Doesn't the filesystem not have to be
uncompressed to install? Wouldn't it already be uncompressed during apt
use?

I understand about rolling out in a way to see what possible problems arise
before implementing defaults.


Thank you


On Mon, Oct 10, 2022, 6:56 AM David Kalnischkies 
wrote:

> On Sun, Oct 09, 2022 at 01:21:45PM -0700, xaq xaq wrote:
> > To simplify a frequently seen halt of `apt upgrade` I have this
> example.. I
> > had to remove keepsolidinc.list
> > from /etc/apt/sources.list.d. Then apt update proceeds. Here is the
> error I
> > get with that source list in place:
> >
> > *Hit:1 http://archive.ubuntu.com/ubuntu
> >  - InRelease
> > Hit:2 http://security.ubuntu.com/ubuntu
> >  --security InRelease
> > Hit:3 http://archive.ubuntu.com/ubuntu
> >  --updates InRelease
> > Hit:4 http://archive.ubuntu.com/ubuntu
> >  --backports InRelease
> > Ign:5 http://apt.keepsolid.com/ubuntu
> >  - InRelease
> > Err:6 http://apt.keepsolid.com/ubuntu
> >  - Release
> >   404  Not Found [IP: 144.217.71.199 80]
> > Reading package lists... Done
> > E: The repository 'http://apt.keepsolid.com/ubuntu
> >  - Release' does not have a Release
> > file.
> > N: Updating from such a repository can't be done securely, and is
> > therefore disabled by default.
> > N: See apt-secure(8) manpage for repository creation and user
> > configuration details.*
>
> What do you mean with "halt of `apt upgrade`"? You have shown us output
> of 'apt update'? And that output suggests that the other archives were
> updated (just that they were already current: cache "Hit").
>
> I suppose if the old data still on disk for keepsolid is indicating
> newer versions of packages than you have installed an upgrade would try
> to download them potentially running also into 404 errors (assuming this
> repo is entirely offline). That would indeed "block" upgrades of other
> packages not from that repo, but what is apt supposed to do instead…
> It is asked to upgrade all packages and can't: Partial upgrades are
> technically supported, but hardly tested in practice as it is
> a combinatorial explosion if you tried. So in theory you upgrade from
> a perfectly supported system with updates available to a system with
> unknown support status and still more updates available. That doesn't
> look like a good default move.
>
> You can tell apt to carry on with --fix-missing if that is what you
> desire, but really, you should either remove that repository (and the
> packages coming from it) for good or work with its provider on fixing
> it/making it available again. After all, --fix-missing lets you carry
> on in a pinch, but it remains that the repository fails to serve updates
> to you, which in the worst case means public known security issues remain
> exploitable on your system.
>
> (This could be part of an attack. It usually isn't, but it is
>  indistinguishable from a MITM blocking access to a repository so
>  that you can not get the security updates the attacker wants to
>  exploit. There are better ways, hence its unlikely, but it is just
>  a less good way as it 

Re: Evolving away from source package realms

[Scott Kitterman]

On October 10, 2022 7:56:07 AM UTC, Gerardo Ballabio 
 wrote:
>Didier Raboud wrote:
>> The last aspect would also be to completely remove the source-package-level
>realms; within a subset, there would be no package-specific maintainers or
>vetoes; disputes would move "out" from source-package-level to subset-level.
>
>Uhm. This makes me wonder what the real goal of this proposal is.
>
>Is it about restricting the ability of DDs to upload any package
>(something that has never really caused any real issues so far as I'm
>aware)? Or is it really about having a way to work around
>uncollaborative maintainers of specific packages?
>
>If it's the latter, I'm afraid it could have more negative effects
>than positive. While "package-specific maintainership" does have its
>problems, it is essentially what has been keeping Debian working until
>now. People take care of their packages because, well, they're their
>packages. If packages aren't assigned to maintainers anymore, I fear a
>situation of "it's everybody's responsibility, therefore it's nobody's
>responsibility".
>
>There are in fact many team-maintained packages, and that's working
>well, but it works because people *voluntarily* agreed to collective
>maintainership, and those teams are usually rather small anyway, with
>an even smaller number of people taking the lead. Those will still
>have veto power.

I think this is generally correct.  Ultimately, I think moving in the suggested 
direction will ultimately add bureaucracy and take away motivation.

Today, who decides a technical question is relatively straightforward.  In the 
first instance the maintainer (or maintainers + uploaders) decides and people 
who disagree have the option to escalate to the tech ctte.  If we do away with 
maintainers, everything will either be free for all revert wars or some other 
structure will be needed to make decisions.  I don't find the idea of doing the 
same work with additional mandatory bureaucracy at all appealing.

There are circumstances within the archive where having more structure makes 
sense and that's where teams have naturally formed.

I deal with more than enough process and procedure when I'm paid to put up with 
it.  Let's not further bureaucratize Debian.

Scott K

Re: Apt feature request / suggestion

[David Kalnischkies]

On Sun, Oct 09, 2022 at 01:21:45PM -0700, xaq xaq wrote:
> To simplify a frequently seen halt of `apt upgrade` I have this example.. I
> had to remove keepsolidinc.list
> from /etc/apt/sources.list.d. Then apt update proceeds. Here is the error I
> get with that source list in place:
> 
> *Hit:1 http://archive.ubuntu.com/ubuntu
>  - InRelease
> Hit:2 http://security.ubuntu.com/ubuntu
>  --security InRelease
> Hit:3 http://archive.ubuntu.com/ubuntu
>  --updates InRelease
> Hit:4 http://archive.ubuntu.com/ubuntu
>  --backports InRelease
> Ign:5 http://apt.keepsolid.com/ubuntu
>  - InRelease
> Err:6 http://apt.keepsolid.com/ubuntu
>  - Release
>   404  Not Found [IP: 144.217.71.199 80]
> Reading package lists... Done
> E: The repository 'http://apt.keepsolid.com/ubuntu
>  - Release' does not have a Release
> file.
> N: Updating from such a repository can't be done securely, and is
> therefore disabled by default.
> N: See apt-secure(8) manpage for repository creation and user
> configuration details.*

What do you mean with "halt of `apt upgrade`"? You have shown us output
of 'apt update'? And that output suggests that the other archives were
updated (just that they were already current: cache "Hit").

I suppose if the old data still on disk for keepsolid is indicating
newer versions of packages than you have installed an upgrade would try
to download them potentially running also into 404 errors (assuming this
repo is entirely offline). That would indeed "block" upgrades of other
packages not from that repo, but what is apt supposed to do instead…
It is asked to upgrade all packages and can't: Partial upgrades are
technically supported, but hardly tested in practice as it is
a combinatorial explosion if you tried. So in theory you upgrade from
a perfectly supported system with updates available to a system with
unknown support status and still more updates available. That doesn't
look like a good default move.

You can tell apt to carry on with --fix-missing if that is what you
desire, but really, you should either remove that repository (and the
packages coming from it) for good or work with its provider on fixing
it/making it available again. After all, --fix-missing lets you carry
on in a pinch, but it remains that the repository fails to serve updates
to you, which in the worst case means public known security issues remain
exploitable on your system.

(This could be part of an attack. It usually isn't, but it is
 indistinguishable from a MITM blocking access to a repository so
 that you can not get the security updates the attacker wants to
 exploit. There are better ways, hence its unlikely, but it is just
 a less good way as it shows errors grabbing attention that something
 is wrong. If apt wouldn't, this would be really easy to use.)

> For the size estimation, I understand it is an estimation, and there could
> be guesswork, but I would posit that if the estimate is even roughly
> correct, a user most likely will not want to proceed, because the
> consequences greatly outweigh the benefits of proceeding.

As I tried to explain already apt downloads itself at the expense of 1.5
MB. It will take up installed 4.5 MB. It hardly changes size from
version to version, so more space freed/used after the upgrade amounts
to more or less 0 MB. Running and using apt requires additional data
which easily takes up 200 MB. Lets assume every package is similar to
apt (they aren't, you e.g. have things like texlive which comes in a
couple hundred MBs of download already). A typical upgrade can contain
more than 1000 packages, so which value between 0 and 200 GB are we
gonna use as a rough estimate for that upgrade? And again, which
partitions are we gonna talk about if you have multiple, like e.g.
a separate /boot ?

APT in this scenario picks 1.5 GB (the size it will need to download all
the packages) on the partition housing its download location:
apt's free space check for the download is done against free blocks
available for everyone. In a default configuration root has an additional
5% of a partition as free space available for their exclusive use.

If your idea is that lets say apt should not continue if you don't have
100 MB free on /boot and 5 GB on / (should be enough, right?) that is
fine for your system and pretty easy to implement with e.g. a
dpkg::pre-invoke hook (no need for the heavier dpkg::pre-install-pkgs
hook). That wont work in the general case though… there are systems
which don't even have 5 GBs of disk space to begin with.

(Did you think apts 1.5 GB is a reasonable low bound? For some people
 that is already way too high! Like if your filesystem supports
 compression)


That script you could package up and make available for everyone who
needs it. Probably 

Re: Evolving away from source package realms

[Gerardo Ballabio]

Didier Raboud wrote:
> The last aspect would also be to completely remove the source-package-level
realms; within a subset, there would be no package-specific maintainers or
vetoes; disputes would move "out" from source-package-level to subset-level.

Uhm. This makes me wonder what the real goal of this proposal is.

Is it about restricting the ability of DDs to upload any package
(something that has never really caused any real issues so far as I'm
aware)? Or is it really about having a way to work around
uncollaborative maintainers of specific packages?

If it's the latter, I'm afraid it could have more negative effects
than positive. While "package-specific maintainership" does have its
problems, it is essentially what has been keeping Debian working until
now. People take care of their packages because, well, they're their
packages. If packages aren't assigned to maintainers anymore, I fear a
situation of "it's everybody's responsibility, therefore it's nobody's
responsibility".

There are in fact many team-maintained packages, and that's working
well, but it works because people *voluntarily* agreed to collective
maintainership, and those teams are usually rather small anyway, with
an even smaller number of people taking the lead. Those will still
have veto power.

Gerardo