Bug#782501: apt-get clean deletes temporary files written by a running apt-get update

[David Kalnischkies]

On Wed, Apr 15, 2015 at 03:54:36PM +0200, Julian Andres Klode wrote:
 Am 15.04.2015 15:48 schrieb Tim Landscheidt t...@tim-landscheidt.de:
  That may unnecessarily narrow down potential fixes for this
  bug.  Of course it would be possible as well (and maybe
  preferable) for apt-get update (and other commands?) to
  create their temporary files elsewhere or, depending on com-
  patibility and privilege considerations, use linkat(2) with
  AT_EMPTY_PATH.

This flag is Linux-specific. If we fix that we shouldn't leave our
hurd and kfreebsd friends out in the cold if possible…
(Its friend open(2) O_TMPFILE is even linux and filesystem-specific)


 Locking seems like the best idea IMO, not sure how things currently work
 for cache though.

The cachefiles are created by pkgCacheGenerator::MakeStatusCache in
apt-pkg/pkgcachegen.cc. Adding some write-locking here and trying to get
one in pkgCacheFile::RemoveCaches in apt-pkg/cachefile.cc and if not
skipping the file should be possible and enough to fix that, right?

So, Tim, you seem to be interested in seeing this fixed in the future,
may I interest you in writing a patch?  Feel free to ask me (or the
others) by mail or IRC (#debian-apt) anything (optionally even stuff
related to apt). ;)


Best regards

David Kalnischkies


signature.asc
Description: Digital signature

Bug#782501: apt-get clean deletes temporary files written by a running apt-get update

[Tim Landscheidt]

I wrote:

 [...]

 IMHO apt-get should not delete temporary files owned by another
 process.

 [...]

That may unnecessarily narrow down potential fixes for this
bug.  Of course it would be possible as well (and maybe
preferable) for apt-get update (and other commands?) to
create their temporary files elsewhere or, depending on com-
patibility and privilege considerations, use linkat(2) with
AT_EMPTY_PATH.

Tim


-- 
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org

Bug#782501: apt-get clean deletes temporary files written by a running apt-get update

[Julian Andres Klode]

Am 15.04.2015 15:48 schrieb Tim Landscheidt t...@tim-landscheidt.de:

 I wrote:

  [...]

  IMHO apt-get should not delete temporary files owned by another
  process.

  [...]

 That may unnecessarily narrow down potential fixes for this
 bug.  Of course it would be possible as well (and maybe
 preferable) for apt-get update (and other commands?) to
 create their temporary files elsewhere or, depending on com-
 patibility and privilege considerations, use linkat(2) with
 AT_EMPTY_PATH.


Clean specifically includes the temporary files, so elsewhere is not an
option. Linkat would be nice in theory, but complicates the code even more.

Locking seems like the best idea IMO, not sure how things currently work
for cache though.

Bug#782501: apt-get clean deletes temporary files written by a running apt-get update

[Tim Landscheidt]

Package: apt
Version: 1.0.9.7
Severity: normal

Dear Maintainer,

unattended-upgrades regularly complained about
(cf. 
https://bugs.launchpad.net/ubuntu/+source/unattended-upgrades/+bug/1366708):

| E:Problem renaming the file /var/cache/apt/pkgcache.bin.WfLyE7 to 
/var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| W:You may want to run apt-get update to correct these problems

Googling brought up
https://bugs.launchpad.net/landscape-client/+bug/1091137:

| When the landscape-client runs apt-get update, it can return an error like 
this:

| E: Problem renaming the file /var/cache/apt/pkgcache.bin.PGPOhV to 
/var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| W: You may want to run apt-get update to correct these problems

| This can happen if someone is running apt-get clean, while the update is 
happening. apt-get clean
| removes all pkgcache.bin files, including the temporary ones that apt-get 
update uses
| to rebuild the cache. This error is harmless, since pkgcache.bin should also 
have been
| removed, and thus the next time the cache will be opened the cache will be 
rebuilt.

| I.e, we should ignore the error above as to not cause intermittent alerts in 
the Landscape
| server, that the user can't do anything to resolve.

And indeed the error is reproducable by:

| scfc@toolsbeta-jessie3:~$ while sudo apt-get clean; do :; done  sudo apt-get 
update; kill -HUP %%
| [1] 6241
| Hit http://apt.wikimedia.org jessie-wikimedia InRelease
| Hit http://mirrors.wikimedia.org jessie InRelease 
| Hit http://mirrors.wikimedia.org jessie-updates InRelease 
  
| Hit http://apt.wikimedia.org jessie-wikimedia/main Sources
  
| Hit http://apt.wikimedia.org jessie-wikimedia/backports Sources   
  
| Get:1 http://mirrors.wikimedia.org jessie/main Sources/DiffIndex [7,876 B]
| Hit http://security.debian.org jessie/updates InRelease 
| Hit http://apt.wikimedia.org jessie-wikimedia/thirdparty Sources
| Hit http://apt.wikimedia.org jessie-wikimedia/main amd64 Packages   
| Hit http://apt.wikimedia.org jessie-wikimedia/backports amd64 Packages  
| Hit http://apt.wikimedia.org jessie-wikimedia/thirdparty amd64 Packages 
| Get:2 http://mirrors.wikimedia.org jessie/main amd64 Packages/DiffIndex 
[7,876 B]
| Get:3 http://mirrors.wikimedia.org jessie/main Translation-en/DiffIndex 
[7,876 B]
| Hit http://mirrors.wikimedia.org jessie-updates/main Sources
| Hit http://mirrors.wikimedia.org jessie-updates/main amd64 Packages
| Hit http://mirrors.wikimedia.org jessie-updates/main Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/backports Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/backports Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/main Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/main Translation-en
| Ign http://apt.wikimedia.org jessie-wikimedia/thirdparty Translation-en_US
| Ign http://apt.wikimedia.org jessie-wikimedia/thirdparty Translation-en
| Hit http://security.debian.org jessie/updates/main Sources
| Hit http://security.debian.org jessie/updates/main amd64 Packages
| Hit http://security.debian.org jessie/updates/main Translation-en
| Fetched 23.6 kB in 1s (12.2 kB/s)
| Problem renaming the file /var/cache/apt/srcpkgcache.bin.ZDdbP3 to 
/var/cache/apt/srcpkgcache.bin - rename (2: No such file or directory)
| Problem renaming the file /var/cache/apt/pkgcache.bin.93xAUQ to 
/var/cache/apt/pkgcache.bin - rename (2: No such file or directory)
| The package lists or status file could not be parsed or opened.
| Can't call method policy on an undefined value at 
/usr/bin/apt-show-versions line 56.
| E: Problem executing scripts APT::Update::Post-Invoke-Success 'test -x 
/usr/bin/apt-show-versions || exit 0 ; apt-show-versions -i'
| E: Sub-process returned an error code
| scfc@toolsbeta-jessie3:~$

IMHO apt-get should not delete temporary files owned by another
process.


-- Package-specific info:

-- apt-config dump --

APT ;
APT::Architecture amd64;
APT::Build-Essential ;
APT::Build-Essential:: build-essential;
APT::Install-Recommends 0;
APT::Install-Suggests 0;
APT::NeverAutoRemove ;
APT::NeverAutoRemove:: ^firmware-linux.*;
APT::NeverAutoRemove:: ^linux-firmware$;
APT::NeverAutoRemove:: ^linux-image-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^linux-headers-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^linux-image-extra-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^linux-signed-image-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^kfreebsd-image-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^kfreebsd-headers-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^gnumach-image-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^.*-modules-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^.*-kernel-3\.16\.0-4-amd64$;
APT::NeverAutoRemove:: ^linux-backports-modules-.*-3\.16\.0-4-amd64$;
APT::NeverAutoRemove::