> On 2021/01/06 12:03, Stuart Henderson wrote: > Looking at this it's better than I thought it would be, there are some > problems though - >
Hey thanks! > - The version number comparison using mcuadros/go-version is wrong, > it doesn't match packages-specs(5). > I took the time to learn some perl yesterday, and holy moly my version comparison code was *very* wrong. Thanks for taking the time to point that out. As a result, I went through and mirrored the perl code as closely as I could to ensure that it matches what OpenBSD does. > - There doesn't seem to be a way to validate that index.pkgup.gz is done > against the current available package build. For this I would suggest > recording the timestamp of the @digital-signature on the quirks package > in the index, and verifying when the update is run. (grep out of > "PKG_DBDIR=/var/empty PKG_PATH=$whatever pkg_info -f quirks" will do > the trick). > Added. I'm parsing the signify block in pure Go (instead of shelling out to pkg_info) because I want to be able to use the index generation code on any Go-supported platform. My own mirror (and from what I understand, some of OpenBSD's own mirrors) aren't necessarily running OpenBSD. > Between those two it could cause problems because the user may try to > update a too-small subset of packages. The first problem is obvious. > The second problem, if a library is bumped after the index is generated, > the required updates won't show up. For both if people use it and then > run into problems it's likely the bug reports will end up with openbsd > rather than pkgup. This makes me not want to add it to packages yet > (adding it could easily be seen as an endorsement of using it). > This would be less of a problem if it at least tries to detect outdated > caches and prints a clear warning. > I hope that my above two fixes rectify this situation in your mind. > Less important but I'd be happier if it used the signature from pkg_info > -qS rather than its own version using grep on +CONTENTS, to guard > against possible future changes to things that pkg_add considers when > deciding whether to update (also I think it would make sense to include > the whole string rather than a hash of the signature, there's no need to > hide that), as long as the full url/filename is used pkg_add will fetch > the file directly without grabbing the index first. i.e. > PKG_DBDIR=/var/empty pkg_info -qS > http://mirror/pub/OpenBSD/snapshots/packages/amd64/moo-1.5p0.tgz > I would like this as well. The problem is that pkg_info -qS is slow. It takes orders of magnitude more time to run than my current signature generation code. I can currently build a complete index from a remote mirror in less than ten minutes. If I switched to using pkg_info, it would take several hours by my math. In addition, I would like to keep genpkgup able to be run on any OS that Go supports instead of only OpenBSD machines. I went ahead and sorta-implemented your suggestion though by matching OpenBSD's current signature format. No more hashes. I was torn on this before, but I actually like your approach better because A: it's easier to debug when things go wrong and B: it's much less CPU-intensive not having to do sha256 stuff. So again, thank you for the recommendation. If the signature format changes in the future, I will gladly update my code to match, or revisit the problem if necessary. Again, thank you for taking the time to look at my tool!
