Hi, Chris Marusich <cmmarus...@gmail.com> skribis:
> The reason Guix is failing to GC dead items in the Docker container is > because those dead items are not on the "top layer", so Docker returns > an EXDEV error: > > https://docs.docker.com/storage/storagedriver/overlayfs-driver/ > > "Renaming directories: Calling rename(2) for a directory is allowed only > when both the source and the destination path are on the top > layer. Otherwise, it returns EXDEV error ('cross-device link not > permitted'). Your application needs to be designed to handle EXDEV and > fall back to a 'copy and unlink' strategy." > > You can observe this by running guix-daemon with strace in the > container, and watching what happens when you try to delete one of the > offending store items (make sure it is a directory). For example: > > 685 rename("/gnu/store/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database", > "/gnu/store/trash/xib50iqk3w1gw9l770mad59m9bi3bcpc-manual-database") = -1 > EXDEV (Invalid cross-device link) > > In most cases, when guix-daemon GC's a dead directory, it does this > (see: nix/libstore/gc.cc): > > - Create a trash directory (usually /gnu/store/trash) > - Move dead directories into the trash directory. > - Delete the trash directory. > > The trash directory is on the "top layer" because it gets created in the > running container. However, in practice many store items from lower > layers are made dead when Stephen's script runs "guix pull" and deletes > the old profiles. If any of those store items were directories, > guix-daemon will fail to GC them because of an XDEV error. If this is > confusing to you, I suggest you experiment with Docker a little bit, and > look closely at the steps that Stephen's script is running. I outlined > this in the email I accidentally deleted, but I'm a little too tired to > reproduce it all a second time. I hope you'll understand. Interesting, thanks for the analysis! > Should Guix do anything about this? We could change guix-daemon to take > correct action in the face of an XDEV error. We could also improve the > logging, since currently it silently swallows the XDEV error. I guess we could delete recursively right away upon EXDEV. It should be just two lines of code, right? > To work around the issue, Stephen can build the images from the same > base image, rather than daisy-chaining new images from old ones. That > way, they would not accumulate layers without bound. Maybe that too. Thanks, Ludo’.