On Sat, 26 Sep 2020 12:14:38 +0200 Johannes Schauer <jo...@debian.org> wrote: > Hi, > > Quoting Josh Triplett (2020-09-26 11:03:12) > > On Sat, Sep 26, 2020 at 07:28:57AM +0200, Johannes Schauer wrote: > > > Quoting Josh Triplett (2020-09-26 06:28:18) > > > > mmdebstrap seems to re-download packages every time it runs. I'd love > > > > to have > > > > a way to cache those packages, so that it can run substantially faster > > > > the > > > > second and subsequent times. > > > > > > Do one thing and do it well. > > > > > > I'm sure you are aware of apt-cacher or apt-cacher-ng? > > > > > > In other situation, for example the mmdebstrap testsuite, I built my own > > > caching mechanism: > > > > > > https://sources.debian.org/src/mmdebstrap/0.7.1-1/make_mirror.sh/ > > > > > > Adding a cache is so simple, you can use a snippet of Python from this > > > pull > > > request of mine: > > > > > > https://salsa.debian.org/debian/devscripts/-/blob/d1c6f2f6a3dca51b77a4a9b37c7d68c1d69270dd/scripts/debbisect#L113 > > > > > > But why add it to mmdebstrap? That just adds additional complexity and > > > feature > > > creep. It makes an already complex tool even more complex. Why not keep > > > this > > > functionality in an additional piece of software? > > > > > > Please convince me. > > > > I'm definitely not proposing integrating a caching proxy of any kind > > into mmdebstrap; that should absolutely be a separate piece of software. > > But a caching proxy doesn't solve the problem for me. Using a caching > > proxy would require one or both of: > > - Using http, rather than https > > - Configuring the target system to use the proxy directly in its config > > > > What I was hoping to see integrated into mmdebstrap would be support for > > just directly copying the relevant files into the target's > > /var/cache/apt, where apt could use them rather than re-downloading > > them. Such files could either come from the host system's > > /var/cache/apt, or from a directory given to mmdebstrap as a > > command-line option (where the latter would also allow mmdebstrap to > > store files from the target's /var/cache/apt before deleting them). > > > > I'm hoping that has the right combination of "sufficiently narrowly > > scoped" and "requires integration" to make sense as a part of > > mmdebstrap. > > so... you want something like this: > > $ mmdebstrap --customize-hook='sync-out /var/cache/apt/archives ./cache' > unstable /dev/null > $ mmdebstrap --variant=apt --setup-hook='mkdir -p > "$1"/var/cache/apt/archives' --setup-hook='sync-in ./cache > /var/cache/apt/archives' unstable output.tar > > The first command fills your local directory "./cache" with the contents of > /var/cache/apt/archives from the first run while the second invocation gets > the > contents from that directory and thus is able to operate a bit faster.
By the time customize-hook runs, mmdebstrap has already cleared /var/cache/apt/archives, so there's nothing to copy out. (Also, I'd prefer to use hardlinks, to avoid making unnecessary copies.) And it appears that copying the cache in via setup-hook does not end up taking effect; either something is clearing/invalidating the cache by the time apt runs, or apt isn't using it. I'd be happy to do this with hooks, but at the moment, I don't see a way to do so successfully.
