(Cc-ing Adam for DSA following the ping about disk space earlier.) Cyril Brulebois <k...@debian.org> (2023-06-13): > Spotted while working on #1037479: we have a bunch of files around that > are no longer useful, because relevant suites are EOL and archived, and > have been dropped from the relevant config files, templates, etc. > > They could probably be cleaned up, but I don't want to rush this kind of > things, hence this bug report.
Apparently we're going to have to do something soon, since the recent updates in picconi have led to a noticeable and worrisome bump in storage used: /dev/mapper/vg0-srv 246G 234G 822M 100% /srv Some cleanup happened in user directories, but I suppose it would make sense to keep track of our disk space usage: /dev/mapper/vg0-srv 246G 136G 99G 58% /srv And beware, there are peaks during runs, with e.g. tmp/ filing up. Also, the mirror (pkgmirror-csail) should like this as well given: /dev/vdb 148G 129G 12G 92% /srv See also graphs (with the usual guest authentication): https://munin.debian.org/debian.org/picconi.debian.org/df.html https://munin.debian.org/debian.org/pkgmirror-csail.debian.org/df.html Our directory looked like this when I started: 426M ./archive 200K ./bin 268K ./cache 8.0K ./cgi-bin 29M ./conf 64K ./cron.d 8.0K ./debian 116G ./files 16M ./.git 133M ./home 300K ./lib 14M ./mail 8.4G ./mirror 4.0K ./po 196K ./static 116K ./templates 7.5G ./tmp 2.6G ./www ------------------- 135G total I was thinking of getting rid of these: $ du -sch $(find -name 'jessie*') 65M ./mirror/202306141247/www/jessie-backports 17M ./mirror/202306141247/www/source/jessie-backports 87M ./mirror/202306141247/www/source/jessie 3.5M ./mirror/202306141247/www/source/jessie-updates 5.3M ./mirror/202306141247/www/source/jessie-backports-sloppy 505M ./mirror/202306141247/www/jessie 12M ./mirror/202306141247/www/jessie-updates 6.4M ./mirror/202306141247/www/jessie-backports-sloppy 196K ./archive/backports/jessie-backports 26M ./www/jessie-backports 5.8M ./www/source/jessie-backports 11M ./www/source/jessie 2.3M ./www/source/jessie-updates 3.4M ./www/source/jessie-backports-sloppy 176M ./www/jessie 3.1M ./www/jessie-updates 4.0M ./www/jessie-backports-sloppy ---------------------------------------------------------------- 930M total and these: $ du -sch $(find -name 'stretch*') 85M ./mirror/202306141247/www/source/stretch 5.1M ./mirror/202306141247/www/source/stretch-backports-sloppy 15M ./mirror/202306141247/www/source/stretch-backports 3.4M ./mirror/202306141247/www/source/stretch-updates 416M ./mirror/202306141247/www/stretch 6.2M ./mirror/202306141247/www/stretch-backports-sloppy 52M ./mirror/202306141247/www/stretch-backports 4.9M ./mirror/202306141247/www/stretch-updates 11M ./www/source/stretch 3.4M ./www/source/stretch-backports-sloppy 5.8M ./www/source/stretch-backports 2.2M ./www/source/stretch-updates 179M ./www/stretch 3.9M ./www/stretch-backports-sloppy 22M ./www/stretch-backports 2.6M ./www/stretch-updates ----------------------------------------------------------------- 813M total But those easy ones would only account for 1.7G combined, as mentioned yesterday. Searching for files containing but not starting with either jessie or stretch, there are a lot of other things that looked like they could go away: - filelists_* - filenames_* - new_package_info_* - package_names_* - packages_all_* - reverse_* - source_names_* - sources_all_* and I expected no collateral damages. To be on the safe side, I created two tarballs, one for jessie, one for stretch, containing all those files, before removing them. I've compressed them so that they take less space than actual files, moved them in a directory that shouldn't be mirrored to pkgmirror-csail, and I've scheduled their deletions in 6 months (via at), which should leave plenty of time to restore them if needed, while making sure full cleanup happens eventually. pkg_user@picconi:/srv/packages.debian.org$ echo 'rm -rv /srv/packages.debian.org/obsolete/' | at 'now + 6 months' warning: commands will be executed using /bin/sh job 5 at Thu Dec 14 18:56:00 2023 pkg_user@picconi:/srv/packages.debian.org$ atq 5 Thu Dec 14 18:56:00 2023 a pkg_user Also removed, a long obsolete file: -rw------- 1 pkg_user pkg_maint 7.5G May 2 2021 tmp/sort01fAge Not sure why this file stays behind, but at least the name is stable, so the non-freed space is getting reclaimed when the next run starts: -rw-r--r-- 1 pkg_user pkg_maint 14794575872 Jun 14 17:37 tmp/sid.sorted This is rather strange since the commit adding this temporary file aimed at removing it properly, and should error out if it fails to do so (which cron logs claim is not happening): https://salsa.debian.org/webmaster-team/packages/-/commit/1b065791a50c7a0e606f2577bcae02b7c5aa190b At the moment, I'm not sure the file deletion (jessie, stretch) is going to be mirrored to pkgmirror-csail automatically. A sync just finished, but it might have started before the actual deletion happened. Let's keep an eye on this after the next oneā¦ Cheers, -- Cyril Brulebois (k...@debian.org) <https://debamax.com/> D-I release manager -- Release team member -- Freelance Consultant
signature.asc
Description: PGP signature