(obviously post unlock)
On 2026/09/30 19:23, Marc Espie wrote:
> On Wed, Sep 30, 2026 at 01:44:41PM +0100, Stuart Henderson wrote:
> >
> > ...however, by_cipher is only used by ports infrastructure when fetching
> > files from ftp.openbsd.org etc, or when dpb fetches files. Specifically
> > when _building_, ports only looks in ${FULLDISTDIR}/file, not
> > ${DISTDIR}/by_ciphers.
I was wrong here, it's not normally used for fetching from ftp.o.o as
a backup SITES - I think it's actually only used for REFETCH=true,
which is currently broken. It tries to fetch from a url like
https://<...>/pub/OpenBSD/distfiles/by_cipher/sha256/xx/xxx<...>xxx=/
i.e. with a trailing / and no filename, so AFAICT the files in by_cipher
on the mirrors are not used at all by ports infra.
But now I've seen what REFETCH is for, it doesn't seem a bad idea to
make it work again, it could simplify diffing old and new files when
upstream re-rolls something, which probably happens more these days than
when it was first invented.
Here's an example of how it could look, just touching link-checksums
and bsd.port.mk so far. Since current files are not used anyway I went
with my idea of .../sha256/xx/xxxxxx as a file rather than a dir which
should be a lot nicer to servers, and s,/,_,g.
Obviously dpb and clean-old-distfiles needed too if this is a sensible
path to take...
Index: fetch/link-checksums
===================================================================
RCS file: /cvs/ports/infrastructure/fetch/link-checksums,v
diff -u -p -r1.4 link-checksums
--- fetch/link-checksums 29 Jul 2008 18:00:19 -0000 1.4
+++ fetch/link-checksums 1 Oct 2026 10:51:16 -0000
@@ -10,12 +10,12 @@
DISTDIR=${DISTDIR:-/usr/ports/distfiles}
cd ${DISTDIR}
-CIPHERS=${CIPHERS:-sha1 md5 rmd160 sha256}
+HASHES=${HASHES:-sha256}
-bdir=by_cipher
-# Build the find so that existing cipher dirs are avoided
+bdir=by_hash
+# Build the find so that existing hash dirs are avoided
exclude="-type d -name $bdir -prune"
-for ci in ${CIPHERS}
+for ci in ${HASHES}
do
exclude="${exclude} -o -type d -name $ci -prune"
done
@@ -23,12 +23,12 @@ echo "find . \( $exclude \) -o -type f -
find . \( $exclude \) -o -type f -print | while read i
do
file=`basename $i`
- for ci in ${CIPHERS}
+ for ci in ${HASHES}
do
- result=`cksum -a $ci -b <$i`
- res=`echo $result|sed -e 's,\(..\).*,\1,'`
- dir=$bdir/$ci/$res/$result
+ result=`cksum -a $ci -b <$i | sed -e 's,/,_,g'`
+ res=`echo $result | sed -e 's,\(..\).*,\1,'`
+ dir=$bdir/$ci/$res
mkdir -p $dir
- ln -f $i $dir/$file
+ ln -f $i $dir/$result
done
done
Index: mk/bsd.port.mk
===================================================================
RCS file: /cvs/ports/infrastructure/mk/bsd.port.mk,v
diff -u -p -r1.1652 bsd.port.mk
--- mk/bsd.port.mk 19 Sep 2026 23:17:29 -0000 1.1652
+++ mk/bsd.port.mk 1 Oct 2026 10:51:16 -0000
@@ -3293,13 +3293,13 @@ _internal-subpackage: ${_PACKAGE_COOKIES
_REFETCH_INFO ?=
# list is pretty much self explanatory
-.for f cipher value in ${_REFETCH_INFO}
+.for f hash value in ${_REFETCH_INFO}
${DISTDIR}/$f:
@lock=${@:T}.dist; ${_SIMPLE_LOCK}; \
${_PFETCH} install -d ${DISTDIR_MODE} ${@:H}; \
cd ${@:H}; \
[email protected]; \
- for site in
${SITE_OPENBSD:=by_cipher/${cipher}/${value:C/(..).*/\1/}/${value}/}
${SITE_OPENBSD:=${cipher}/${value}/}; do \
+ for site in
${SITE_OPENBSD:=by_hash/${hash}/${value:S,/,_,g:C/(..).*/\1/}/${value:S,/,_,g}};
do \
${ECHO_MSG} ">> Fetch $${site}$u"; \
if ${_PFETCH} ${FETCH_CMD} -o $$file $${site}$u; then \
ck=`${_size_fragment} $$file $p`; \
