On Fri 30/03/2018 23:18, Stuart Henderson wrote: > On 2018/03/30 20:19, Björn Ketelaars wrote: > > On Fri 30/03/2018 18:22, Stuart Henderson wrote: > > > On 2018/03/30 19:01, Björn Ketelaars wrote: > > > > On Fri 30/03/2018 13:45, Klemens Nanni wrote: > > > > > On Fri, Mar 30, 2018 at 05:30:55AM +0200, Björn Ketelaars wrote: > > > > > > zstd received an update, which focuses on performance. Changelog > > > > > > can be > > > > > > found at https://github.com/facebook/zstd/blob/dev/NEWS. The major > > > > > > number of the shared lib has been bumped as a couple of symbols have > > > > > > been removed, and new ones have been added. > > > > > > > > > > > > I've been running with this update for a couple of days on AMD64 > > > > > > using > > > > > > its only customer: borgbackup. > > > > > > > > > > > > OK? > > > > > Has anyone contacted upstream already to merge the trivial > > > > > detection/support bits we're carrying in our patches? Would be nice to > > > > > get those in, effectively limiting required patches on our site to > > > > > library version handling. > > > > > > > > > > Tests fail with a bunch of "use of undeclared identifier", did they > > > > > pass > > > > > before? > > > > > > > > > > fatal error: too many errors emitted, stopping now > > > > > [-ferror-limit=] > > > > > 2 warnings and 20 errors generated. > > > > > 2 warnings and 20 errors generated. > > > > > > > > > > > > I'm having a deja vu [0] ;-) > > > > > > > > Could you send me your test.log? > > > > > > > > [0] https://marc.info/?l=openbsd-ports&m=151473111522358&w=2 > > > > > > > > > > I don't see that here. kn, do you have anything funny in > > > CFLAGS/CDIAGFLAGS/etc? > > > > > > I have a different problem with tests though: > > > > > > - Compress without dictID > > > tmp : 21.56% ( 42430 => 9148 bytes, tmp.zst) > > > > > > tmp.zst : Decoding error (36) : Restored data doesn't match checksum > > > gmake[1]: *** [Makefile:307: zstd-playTests] Error 1 > > > gmake[1]: Leaving directory '/usr/obj/ports/zstd-1.3.4/zstd-1.3.4/tests' > > > > I ran 'make clean && make test' 3 times, one of which gave the same > > error. The other 2 runs were ok. Included a test.log of one of the > > successful runs. > > > > Any suggestions on how to debug this issue? > > Oh, the in-tree one does the same - since it's not worse than that > I'm OK with updating, but maybe see if upstream have any suggestions > about the "without dictID" test?
New diff. zstd requires a stable sort, which libc's qsort() is not guaranteed to be. Upstream proposed to use mergesort(). https://github.com/facebook/zstd/issues/1088 Diff below brings zstd to 1.3.4. I ran 'make test' a couple of times. All runs were successful. OK? Index: Makefile =================================================================== RCS file: /cvs/ports/archivers/zstd/Makefile,v retrieving revision 1.15 diff -u -p -r1.15 Makefile --- Makefile 14 Jan 2018 10:30:28 -0000 1.15 +++ Makefile 4 Apr 2018 13:03:53 -0000 @@ -4,9 +4,8 @@ COMMENT = zstandard fast real-time comp GH_ACCOUNT = facebook GH_PROJECT = zstd -GH_TAGNAME = v1.3.3 -REVISION = 0 -SHARED_LIBS = zstd 0.0 #1.3.3 +GH_TAGNAME = v1.3.4 +SHARED_LIBS = zstd 1.0 # 1.3.4 CATEGORIES = archivers @@ -17,7 +16,7 @@ MAINTAINER = Bjorn Ketelaars <bjorn.ket # BSD 3 clauses + GPLv2 PERMIT_PACKAGE_CDROM = Yes -WANTLIB = c lz4 lzma pthread z +WANTLIB = c lz4 lzma pthread z LIB_DEPENDS = archivers/lz4 \ archivers/xz @@ -27,7 +26,7 @@ COMPILER_LANGS = c MAKE_ENV = CC="${CC}" \ CFLAGS="${CFLAGS}" \ - CPPFLAGS="-I${LOCALBASE}/include" \ + CPPFLAGS="-I${WRKSRC}/lib -I${LOCALBASE}/include" \ CXX="${CXX}" \ CXXFLAGS="${CXXFLAGS}" \ LDFLAGS="-L${LOCALBASE}/lib" \ Index: distinfo =================================================================== RCS file: /cvs/ports/archivers/zstd/distinfo,v retrieving revision 1.9 diff -u -p -r1.9 distinfo --- distinfo 4 Jan 2018 14:47:20 -0000 1.9 +++ distinfo 4 Apr 2018 13:03:53 -0000 @@ -1,2 +1,2 @@ -SHA256 (zstd-1.3.3.tar.gz) = p3xHFT7n3gJibFsqCXAFeGtxaIvmHp+4GAagEfkLKXs= -SIZE (zstd-1.3.3.tar.gz) = 1857232 +SHA256 (zstd-1.3.4.tar.gz) = kuQbbo3Sa71GJI6KodhvFVG8Ihp5Yneuk2KVTybWBak= +SIZE (zstd-1.3.4.tar.gz) = 2059958 Index: patches/patch-Makefile =================================================================== RCS file: /cvs/ports/archivers/zstd/patches/patch-Makefile,v retrieving revision 1.1 diff -u -p -r1.1 patch-Makefile --- patches/patch-Makefile 4 Jan 2018 14:47:20 -0000 1.1 +++ patches/patch-Makefile 4 Apr 2018 13:03:53 -0000 @@ -3,7 +3,7 @@ $OpenBSD: patch-Makefile,v 1.1 2018/01/0 Index: Makefile --- Makefile.orig +++ Makefile -@@ -107,7 +107,7 @@ clean: +@@ -120,7 +120,7 @@ clean: #------------------------------------------------------------------------------ # make install is validated only for Linux, OSX, Hurd and some BSD targets #------------------------------------------------------------------------------ Index: patches/patch-examples_Makefile =================================================================== RCS file: patches/patch-examples_Makefile diff -N patches/patch-examples_Makefile --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ patches/patch-examples_Makefile 4 Apr 2018 13:03:53 -0000 @@ -0,0 +1,14 @@ +$OpenBSD$ + +Index: examples/Makefile +--- examples/Makefile.orig ++++ examples/Makefile +@@ -22,7 +22,7 @@ all: simple_compression simple_decompression \ + multiple_streaming_compression streaming_memory_usage + + $(LIB) : +- make -C ../lib libzstd.a ++ $(MAKE) -C ../lib libzstd.a + + simple_compression : simple_compression.c $(LIB) + $(CC) $(CPPFLAGS) $(CFLAGS) $^ $(LDFLAGS) -o $@ Index: patches/patch-lib_dictBuilder_cover_c =================================================================== RCS file: patches/patch-lib_dictBuilder_cover_c diff -N patches/patch-lib_dictBuilder_cover_c --- /dev/null 1 Jan 1970 00:00:00 -0000 +++ patches/patch-lib_dictBuilder_cover_c 4 Apr 2018 13:03:53 -0000 @@ -0,0 +1,19 @@ +$OpenBSD$ + +zstd doesn't play nice with libc's qsort(). Reason is that qsort() is not +guaranteed to be stable. Proposed fix is to use libc's mergesort() instead. + +https://github.com/facebook/zstd/issues/1088 + +Index: lib/dictBuilder/cover.c +--- lib/dictBuilder/cover.c.orig ++++ lib/dictBuilder/cover.c +@@ -583,7 +583,7 @@ static int COVER_ctx_init(COVER_ctx_t *ctx, const void + } + /* qsort doesn't take an opaque pointer, so pass as a global */ + g_ctx = ctx; +- qsort(ctx->suffix, ctx->suffixSize, sizeof(U32), ++ mergesort(ctx->suffix, ctx->suffixSize, sizeof(U32), + (ctx->d <= 8 ? &COVER_strict_cmp8 : &COVER_strict_cmp)); + } + DISPLAYLEVEL(2, "Computing frequencies\n"); Index: patches/patch-tests_playTests_sh =================================================================== RCS file: /cvs/ports/archivers/zstd/patches/patch-tests_playTests_sh,v retrieving revision 1.8 diff -u -p -r1.8 patch-tests_playTests_sh --- patches/patch-tests_playTests_sh 4 Jan 2018 14:47:20 -0000 1.8 +++ patches/patch-tests_playTests_sh 4 Apr 2018 13:03:53 -0000 @@ -10,7 +10,7 @@ Index: tests/playTests.sh *) MD5SUM="md5sum" ;; esac -@@ -237,7 +238,7 @@ rm ./*.tmp ./*.zstd +@@ -258,7 +259,7 @@ rm ./*.tmp ./*.zstd $ECHO "frame concatenation tests completed"
