I can reproduce this on command, and have two workarounds, (one dirty, one ugly).
I'm using FAI on Debian 9.13. So, the problem is, that when apt writes a translations related file, it is writing a symlink to the extracted .bz2, rather than writing a .bz2 file. Problem reproduction: I am using apt-cacher-ng, and am currently pointing it to archive.debian.org. I am also using apt against a chroot, as part of using the Fully Automated Installer. This is happening while I'm trying to set up a mirror of debian, with only the packages I have selected. as part of that, my apt options are as follows: ``` -o Aptitude::Log=/dev/null -o Aptitude::CmdLine::Ignore-Trust-Violations=yes -o APT::Get::AllowUnauthenticated=true -o DPkg::force-conflicts::=yes -o Dir::State=/usr/fai/mirror/aptcache//var/lib/apt -o Dir::Log=/usr/fai/mirror/aptcache//var/log/apt -o Dir::State::extended_states=/usr/fai/mirror/aptcache//var/lib/apt/lists/extended_states -o Dir::State::status=/usr/fai/mirror/aptcache//statefile -o APT::Get::Force-Yes=true -o Dir::Cache=/usr/fai/mirror/aptcache//var/cache/apt -o Dir::State=/usr/fai/mirror/aptcache//var/cache/apt -o Dir::Cache::Archives=/usr/fai/mirror/aptcache//var/cache/apt/archives -o Dir::Etc=/usr/fai/mirror/aptcache//etc/apt/ -o Dir::State::Lists=/usr/fai/mirror/aptcache//var/lib/apt/lists/ ``` The error in question is easy to produce: ``` faiserver:/home/fai# apt-get $aptoptions update Ign:1 http://127.0.0.1:3142/archive.debian.org/debian stretch InRelease Get:2 http://127.0.0.1:3142/archive.debian.org/debian stretch Release [118 kB] Get:3 http://127.0.0.1:3142/archive.debian.org/debian stretch Release.gpg [3,177 B] Ign:3 http://127.0.0.1:3142/archive.debian.org/debian stretch Release.gpg Hit:5 http://127.0.0.1:3142/archive.debian.org/debian stretch/main Translation-en Ign:5 http://127.0.0.1:3142/archive.debian.org/debian stretch/main Translation-en Hit:4 http://127.0.0.1:3142/archive.debian.org/debian stretch/main i386 Packages Ign:4 http://127.0.0.1:3142/archive.debian.org/debian stretch/main i386 Packages Hit:5 http://127.0.0.1:3142/archive.debian.org/debian stretch/main Translation-en Ign:5 http://127.0.0.1:3142/archive.debian.org/debian stretch/main Translation-en Hit:4 http://127.0.0.1:3142/archive.debian.org/debian stretch/main i386 Packages Ign:4 http://127.0.0.1:3142/archive.debian.org/debian stretch/main i386 Packages Err:5 http://127.0.0.1:3142/archive.debian.org/debian stretch/main Translation-en BZ2_bzread: /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142_archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 Read error (-5: DATA_ERROR_MAGIC) Get:4 http://127.0.0.1:3142/archive.debian.org/debian stretch/main i386 Packages [9,587 kB] Fetched 121 kB in 3s (40.2 kB/s) Reading package lists... Done W: GPG error: http://127.0.0.1:3142/archive.debian.org/debian stretch Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 04EE7237B7D453EC NO_PUBKEY 648ACFD622F3D138 NO_PUBKEY 0E98404D386FA1D9 NO_PUBKEY EF0F382A1A7B6500 W: The repository 'http://127.0.0.1:3142/archive.debian.org/debian stretch Release' is not signed. N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use. N: See apt-secure(8) manpage for repository creation and user configuration details. E: Failed to fetch http://127.0.0.1:3142/archive.debian.org/debian/dists/stretch/main/i18n/Translation-en BZ2_bzread: /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142_archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 Read error (-5: DATA_ERROR_MAGIC) E: Some index files failed to download. They have been ignored, or old ones used instead. faiserver:/home/fai# ``` that's the error in question. A quick inspection of what the file it is complaining looks like is as follows: ``` faiserver:/home/fai# ls -la /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 lrwxrwxrwx 1 root root 122 Jun 8 12:41 /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142_archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 -> /usr/fai/mirror/aptcache/var/lib/apt/lists/127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en faiserver:/home/fai# ls -la /usr/fai/mirror/aptcache/var/lib/apt/lists//127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en -rw-r--r-- 1 root root 26417876 Jul 18 2020 /usr/fai/mirror/aptcache/var/lib/apt/lists//127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en faiserver:/home/fai# file /usr/fai/mirror/aptcache/var/lib/apt/lists//127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en /usr/fai/mirror/aptcache/var/lib/apt/lists//127.0.0.1:3142_archive.debian.org_debian_dists_stretch_main_i18n_Translation-en: UTF-8 Unicode text ``` so apt-get has created a symlink to the extracted form of this file, and is trying to extract it. no good. There are two workarounds: Workaround #1(DO NOT DO THIS): nuke the symlink, and download it yourself: ``` faiserver:/home/fai# rm /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 faiserver:/home/fai# wget -q http://127.0.0.1:3142/archive.debian.org/debian/dists/stretch/main/i18n/Translation-en.bz2 -O /usr/fai/mirror/aptcache/var/lib/apt/lists/partial/127.0.0.1:3142 _archive.debian.org_debian_dists_stretch_main_i18n_Translation-en.bz2 ``` after this, apt-get update works as expected. Workaround #2(much more sane): do not try to download the translations at all! in your sources.list, after the 'deb' part of an entry, add '[ lang=none ]', to prevent apt-get from performing this step. on my system, I go from: ``` deb http://127.0.0.1:3142/archive.debian.org/debian stretch main ``` to: ``` deb [ lang = none ] http://127.0.0.1:3142/archive.debian.org/debian stretch main ``` This prevents apt-get from downloading the translations, and hence prevents it from storing them incorrectly. Let me know if any more information would help.
