On 7/12/22 03:51, Jeff Norden wrote:
<snip>
The core of the idea is to change the ordering of the package archives
so that
the files required for an upgrade will be at the beginning. Ideally, they
would ordered according to the package-version in which they were last
changed. Each pkg.tar file would start with files that are different in
the
current version than in the previous one. The next section would be
those that
are the same in the last two versions but differed in the one previous to
that, etc. This would turn the tar file into something like a multi-volume
incremental backup tape, except that all of the outdated files have been
expunged. There would be no redundant files (and no way to restore
previous
versions). If I'm correct about (A), these files would work fine with
current
pacman/alpm software and be completely backwards compatible. This would
make
it possible to do an incremental upgrade by retrieving just an initial
part of
the archive, which could be done by adding an 'HTTP range' header to the
request.
A client that wants to upgrade foo from p.q-r to x.y-z, would need to
have two
additional pieces of information in order to try this. It would need the
exact size N of an initial portion of the uncompressed foo-x.y-z.pkg.tar
that
contains all blocks for the x.y-z files that differ from the q.r-s ones.
(This initial portion would, in fact, be a valid tar file.) It would also
need a number Nz such that piping the first Nz bytes of the compressed
pkg.tar.zst package file to zstd would produce at least N valid bytes of
output before zstd failed upon receiving an early eof.
If the client decides that Nz is sufficiently smaller than the full
download
size, it would do something like the following:
curl --range 0-Nz https://server:/path/to/foo-x.y-z.pkg.tar.zsd \
| zstd -d 2>/dev/null | head -c N > foo-x.y-z.pkg.part.tar
The client could make its decision on whether this is worthwhile based
on its
network bandwidth, computing resources, etc.
<snip>
Thanks for the very detailed suggestion. I think the downfall is how to
calculate the size needed to download. Pacman/makepkg supports at least
9 compression methods, and I don't think this will be an easy
calculation for any of them.
Allan
