On Tue, Jun 16, 2020 at 04:59:07PM -0400, Jeremy O'Brien wrote: > Hey misc@, > > I wrote a quick little tool here: > https://github.com/neutralinsomniac/obsdpkgup in Go to show available package > upgrades from your configured mirror. > > It takes no more than a few seconds (the time it takes to download index.txt > from the package repo) to show you all packages that have received a version > bump. This tool *won't* show same-version package-rebuild upgrades, so it > shouldn't be used as a complete replacement to running 'pkg_add -u', but > rather as a companion to show when actual newer versions of packages are > released. I just noticed that in my 99% case, I was waiting anywhere from > 5-10 minutes for 'pkg_add -u' to complete checking all ~400 of my installed > packages, and it uses a considerable amount of bandwidth while doing so. > > As I understand it, the pkgtools detect same-version rebuilds by downloading > enough of every installed package tgz to check the metadata contained within > to determine if an upgrade is needed. If anyone knows of an alternative way > to determine when a same-version package install is required, I would love to > know of it. In the meantime, I hope someone else can make use of this tool as > well.
The concept you need to understand is snapshot shearing. A full package snapshot is large enough that it's hard to guarantee that you will have a full snapshot on a mirror at any point in time. In fact, you will sometimes encounter a mix of two snapshots (not that often, recently, but still) Hence, the decision to not have a central index for all packages, but to keep (and trust) the actual meta-info within the packages proper. The only way to avoid that would be to have specific tools for mirroring, and to ask mirror sites to set aside twice as much room so that they can rotate snapshots. Now, each package has all dependency information in the packing-list... which allows pkg_add to know whether to update or not. We do a somewhat strict check, we have tried to relax the check at some point in the past, but this was causing more trouble than it was worth. The amount of data transmitted during pkg_add -u mostly depends on signify: you have to grab at least a full signed block, which is slightly over 64KB. On most modern systems, the bandwidth is not an issue, the number of RTT is more problematic. The way to speed pkg_add -u up would be to keep the connection alive, which means ditching ftp(1) and switching to specific code that talks modern http together with byte-ranges.
