On Mon, Jun 03, 2024 at 08:49:04PM -0700, Steve Langasek wrote: > On Fri, May 03, 2024 at 08:43:11AM +0200, Heinrich Schuchardt wrote: > > On Debian I have seen apt-update downloading diff files. Why don't we use > > those for Ubuntu > > Disclaimer: I have never benchmarked this, but am going by my observations > from 20 years ago when this was adopted by Debian. > > In the common case, pdiff files decrease the *data transfer size* for > downloading indices, but increase the *clock time* it takes to update the > apt database. Therefore there has never been evidence that it's a net win > given modern Internet connections, and no one has ever agitated for this to > be a priority in Ubuntu/Launchpad.
It's much faster these days. Debian also moved to server-side merged pdiffs now, so it's just one file to download and apply, whereas before multiple files were downloaded in the pipeline and then merged locally before being applied. And I guess in the very old days it was not merged at any point. But in any case, this has been brought up a couple of times and the main issue why we don't have pdiffs for Ubuntu is arguably that we don't have like 4 dinstall runs a day, but more like 72 launchpad publisher runs, and having 18x as many deltas is quite expensive both in storage space and calculating them. Which is why a zsync-style synchronization would be better where we compress the file block wise and can fetch changed blocks; however the APT HTTP code is fairly unreliable and that's fairly complex, and you need to use something like zstd with a custom dictionary to make this still compress efficiently. An alternative would be replacing the pdiff format with a tagged one, where each hunk is associated with a timestamp, such that you can then ship one larger patch file per day or so, and you only apply the bits you actually need. Essentially you just need to keep the previous run's pdiff2 around to keep pdiffs working on upgrades, so it will be 2 diffs for the current day, and 1 for each past day or so. I don't believe there is much value in this for stable releases though as has been pointed out before. This may change with LTS. But also we could come up with a much simpler approach where we just split the -updates/-security pockets into 6 months cycles (i.e. add 2024H1 or 24.04.1 subdirectories, add some fancy way to InRelease files to state you need to fetch these subdirectories too) and then you just fetch the current cycle on update. But first fetches would be annoying. But still not convinced that is worth the effort. -- debian developer - deb.li/jak | jak-linux.org - free software dev ubuntu core developer i speak de, en
signature.asc
Description: PGP signature
-- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel