Bug#1068823: Stepwise Debian upgrade to enable systems with little free storage space to upgrade without breaks due to "No space left on device"
On Thu, Apr 11, 2024 at 04:46:03PM +, mYnDstrEAm wrote: > With the two commands above one can already split it up into two steps but > especially the second command still requires a lot of disk space. I am going to assume that your "a lot of disk space" stems from the *.deb files that are downloaded. If so, you can e.g. attach an USB disk/ drive and mount it e.g. under /media/apt-archives Tell apt to use that directory instead of /var/cache/apt/archives, e.g.: apt upgrade -o dir::cache::archives=/media/apt-archives (for some more free MBs you could 'apt clean' and then move dir::cache elsewhere, but for that you need to create some directories in the target location and the binary caches are not THAT large to make it really worthwhile in practice. Similar for other files like /var/lib/apt aka dir::state::lists) Instead of an USB drive you could do the same with e.g. an SD card, drop them into RAM (if your device has surprisingly more RAM than disk) or even use a network share (NFS, sshfs, … you name it). The filesystem is not usually a concern (as in: even fat32 should work given we encode away the : in epochs). Note that whoever has write access to the files on the storage (or in case of unencrypted transfer, also everyone who can meddle with transfer over the network) could use that to attack you as apt (well, apt will casually check them first, but after that and dpkg, who actually interacts with them the most) will assume that the files in /var/cache/apt/archives (or where ever else you stored them and told apt to use them) are valid & trusted. Note also that apt uses for its space check statvfs(3) f_bavail, as in, depending on how you configured your disk, it should have a couple of additional free blocks in reserve (typically 5%, see tune2fs(8) -m). If you know what you are doing, you could decrease that value. Note that the value apt displays is only an estimate, powered by what the individual packages claim (via dpkg), which is an estimate. Also, if you happen to have a 2GB installed, the upgrade will roughly take an additional 2GB as dpkg would first extract the new files along the old ones and then replace them in one swoop – so for a bit, you have that package installed two times. Multiple this by group size, divide by unchanged files and sprinkle some salt over it for flavour. Predictions are hard, especially about the future. I would in general not recommend to try approaches like upgrading individual packages as that easily leads unsuspecting users into situations that nobody else has encountered before: aka bugs in packages that nobody else will encounter as they are either hidden by the involved set usually being upgraded together as intended™ or – which tends to be even worse – the breakage is known but ignored on purpose as the solution is far worse than the problem (at least for everyone doing upgrades the normal way – example: usrmerge). Also, but that is just an aside, people grossly overestimate how easy it is for packages to be upgraded individually (compare: t64 testing migration). Best regards David Kalnischkies signature.asc Description: PGP signature
Bug#1068823: Stepwise Debian upgrade to enable systems with little free storage space to upgrade without breaks due to "No space left on device"
On Thu, 2024-04-11 at 16:46 +, mYnDstrEAm wrote:
>
> For example, I think a good approach to this would be something like
> this (if the user is low on root partition disk space):
> 1. asking for *at least* 400MB to be available
> 2. if a parameter for stepwise is set or it detected low free disk
> space:
> 3. downloading the first 300 MB or so of packages
> 4. installing these
> 5. deleting the cached packages from /var/cache/apt/archives/
> 6. downloading the next batch up to the storage limit set at start
> 7. and so on (without exiting in-between)
>
quick and dirty and not tested:
while apt -s upgrade | grep '^Inst' | head -1 | awk '{print $2}' |
xargs apt install; do apt clean; done
Use head -10 or whatever fits for more/less packages.
--
Bernd ZeimetzDebian GNU/Linux Developer
http://bzed.dehttp://www.debian.org
GPG Fingerprint: ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
Bug#1068823: Stepwise Debian upgrade to enable systems with little free storage space to upgrade without breaks due to "No space left on device"
Package: general A distro upgrade of Debian needs a lot of disk space on the root partition. That partition often doesn't have a large size. That could for example be because it's on a mobile device, on a SSD drive, has lots of installed software, or because the user simply followed the advice of guides that usually recommend the root partition to be relatively small. I had this problem when upgrading to Debian12 and also had it during the last distro upgrade. A problem is that the storage requirements displayed when running sudo apt-get upgrade --without-new-pkgs or sudo apt-get full-upgrade are lower than what is actually needed. Obviously, that shouldn't be the case. Solving this problem could be redundant if there was a way to make it upgrade incrementally in several steps. Could you please make stepwise distribution upgrades possible? With the two commands above one can already split it up into two steps but especially the second command still requires a lot of disk space. That it displays less storage requirements than actually needed makes it more difficult to avoid the problem. Avoiding and solving potential issues resulting from the upgrade breaking or from what the user tries to do to free up disk space requires skill and time, and makes the upgrade userunfriendly and far less smooth than it could be. Here's what I do to free up disk space and I had to do some of the things while the upgrade was ongoing as I noticed that it takes up more disk space than I previously freed up: https://unix.stackexchange.com/q/774199/233262 (such as deleting the cached packages after an unrelated fatal error breaking the upgrade so that it downloaded half of them again when I ran the full-upgrade command again after solving that). This could also be an issue for apt but it could probably also be done independently from that package which is why I filed it under general for now. One could also make it so that it dynamically adjusts depending on how much disk space is available throughout the upgrade. It could and probably should still only be one command. For example, I think a good approach to this would be something like this (if the user is low on root partition disk space): 1. asking for *at least* 400MB to be available 2. if a parameter for stepwise is set or it detected low free disk space: 3. downloading the first 300 MB or so of packages 4. installing these 5. deleting the cached packages from /var/cache/apt/archives/ 6. downloading the next batch up to the storage limit set at start 7. and so on (without exiting in-between)
