Benchmark results are attached. My approach was as follows:
1. Build the latest libjpeg-turbo and libjxl commits
(2dfe6c0fe9e18671105e94f7cbf044d4a1d157e6 and
99663f9d7be605c47554fedc641de407b7095364, respectively) using the same
version of GCC (10.2.1.) For both builds, use a dedicated build
directory under the top level of the source tree.
2. Build a special version of tjbench forJpegli that uses Jpegli's
libjpeg API wrapper:
a. Modify third_party/libjpeg-turbo/turbojpeg.c, replacing the
bodies of tjEncodeYUVPlanes(), tjEncodeYUV3(), tjDecodeYUVPlanes(),
tjDecodeYUV(), and tjTransform() with
return -1;
This eliminates any dependencies between the TurboJPEG API and
internal libjpeg API library functions, which are not exposed in
Jpegli's libjpeg API wrapper. (We don't need YUV encoding/decoding or
lossless transformation in order to evaluate Google's claims regarding
Jpegli.)
b. Modify lib/jpegli.cmake, adding:
configure_file(
../third_party/libjpeg-turbo/jconfigint.h.in
include/jpegli/jconfigint.h)
(jconfigint.h is needed in order to build the TurboJPEG API.)
c. In the libjxl build directory, build tjbench:
gcc -O3 -o tjbench \
-I../third_party/libjpeg-turbo -Ilib/include/jpegli \
-DBMP_SUPPORTED -DPPM_SUPPORTED \
../third_party/libjpeg-turbo/tjbench.c \
../third_party/libjpeg-turbo/tjutil.c \
../third_party/libjpeg-turbo/turbojpeg.c \
../third_party/libjpeg-turbo/jdatadst-tj.c \
../third_party/libjpeg-turbo/jdatasrc-tj.c \
../third_party/libjpeg-turbo/rdbmp.c \
../third_party/libjpeg-turbo/rdppm.c \
../third_party/libjpeg-turbo/wrbmp.c \
../third_party/libjpeg-turbo/wrppm.c \
-Wl,-rpath,`pwd`/lib/jpegli lib/jpegli/libjpeg.so -lm
3. Use the five test images described in this report
https://libjpeg-turbo.org/About/Performance, along with tjbench, to
measure the baseline and progressive compression and decompression
performance of both libjpeg-turbo and Jpegli on my two Linux test
machines. (The Xeon W3530 supports only SSE2 instructions, whereas the
W5820 supports AVX2 instructions.)
Discussion
--
Google makes three claims regarding Jpegli:
1. "Jpegli compresses images more efficiently than traditional JPEG codecs."
Of the five images tested, only nightshot_iso_100.ppm consistently
produced significantly better full-color compression ratios with Jpegli
than with libjpeg-turbo at the tested quality level (95). The other
images produced mixed results, sometimes compressing slightly better
with Jpegli but other times compressing significantly worse. MozJPEG,
by comparison, always produced better compression ratios than
libjpeg-turbo across the board. (Note that MozJPEG always enables
progressive mode, so it was compared only against libjpeg-turbo and
Jpegli with progressive mode enabled.)
2. "Jpegli's coding speed is comparable to traditional approaches, such
as libjpeg-turbo and MozJPEG."
- The compression performance of Jpegli is certainly better, across the
board, than that of MozJPEG. However, once again, this was at the
expense of not always improving the compression ratio relative to
libjpeg-turbo.
- Under no circumstances was the baseline compression performance of
Jpegli comparable to that of libjpeg-turbo. libjpeg-turbo was about
2-8x faster.
- The progressive compression performance of Jpegli was in the ballpark
of libjpeg-turbo and was sometimes better than that of libjpeg-turbo
(particularly on the newer Intel CPU that has AVX2 instructions.)
- Google does not claim that Jpegli's decompression performance is
comparable to that of libjpeg-turbo. They recommend continuing to use
libjpeg-turbo for decompression. Thus, I do not discuss decompression
performance herein. To do so fairly would have required measuring the
decompression performance of images compressed using Jpegli and
decompressed using libjpeg-turbo, which is a bit of a pain.
3. With Jpegli, "images will look clearer and have fewer observable
artifacts."
I did a quick & dirty evaluation of this claim using DSSIM (commit
cb87d9affb937a4dd6d7364b543c39c8723e5494.) Based on those metrics, it
seems plausible to claim that Jpegli (relative to libjpeg-turbo)
produces a better compression ratio with similar perceptual quality, or
that it produces better perceptual quality with a similar compression
ratio, but not that it consistently does both simultaneously. That
makes it problematic as a drop-in replacement for libjpeg[-turbo],
because for a given JPEG quality level, whether Jpegli produces a better
compression ratio or a higher-quality image than libjpeg-turbo seems to
depend on the image.
Potential criticisms of my results:
- I freely admit that the five test images I use for benchmarking
libjpeg-turbo are not a comprehensive representation of photographic
content. That is intentional,
