On 5/16/20 3:38 AM, Paul Spooren wrote: > Could we use that for the SDK/ImageBuilder as well? > > Best, > Paul
Sure, have some benchmark results with the current ath79 snapshot IB: * openwrt-imagebuilder-ath79-generic.Linux-x86_64.tar: 515M * xz -7e (current implementation): 96M, ~100s * zstd -3 (default setting): 248M, ~1s * zstd -19: 112M, ~73s * zstd --ultra -20: 93M, ~97s * zstd --ultra -22: 78M, ~136s So it seems at --ultra -20 we have a sweet spot where we surpass xz -7e in both compression ratio and speed. --ultra just unlocks the highest compression levels >19, at the cost of addional memory use for both compression and decompression. All of the above measurements were made without multithreading. The multithreaded mode of xz (as used in OpenWrt master) makes compression worse, while zstd produces an idential file with and without multithreading: * xz -7e -T32: 113M, ~17s * zstd -19 -T32: 112M: ~12s * zstd --ultra -20 -T32: 93M, ~30s IMO the fact that xz compresses differently depending on the number of threads used is also a big problem, as it makes it harder to make builds fully reproducible. If we switch to zstd for SDK/IB, I would go for level -19: it's significantly faster than --ultra -20, especially in multithreaded mode, and the size difference seems acceptable to me (in fact, our current snapshot builds use xz in multithreaded mode, so the current IB tar.xz on the download server is 113M, not 96M. Matthias
signature.asc
Description: OpenPGP digital signature
_______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
