[
https://issues.apache.org/jira/browse/FLINK-40054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Martijn Visser resolved FLINK-40054.
------------------------------------
Fix Version/s: 2.4.0
Resolution: Fixed
Fixed in apache/flink:master 0ceabd785c4c2db3ed92e06896895b606f6fdcd9
> Update aircompressor to 2.0.3
> -----------------------------
>
> Key: FLINK-40054
> URL: https://issues.apache.org/jira/browse/FLINK-40054
> Project: Flink
> Issue Type: Technical Debt
> Components: Runtime / Network
> Reporter: Martijn Visser
> Assignee: Martijn Visser
> Priority: Major
> Fix For: 2.4.0
>
>
> Flink bundles io.airlift:aircompressor 0.27 in flink-runtime (shaded and
> relocated), where it provides the LZO and ZSTD codecs for network shuffle
> buffer compression (taskmanager.network.compression.codec). The LZ4 codec
> uses lz4-java and is unaffected.
> aircompressor 2.0.3 is the latest release of the Java-8-compatible
> maintenance line (branch release-2.x). It is 0.27 plus backported fixes for
> CVE-2025-67721, an uninitialized-memory data leak in the Snappy and LZ4
> decompressors when the match offset is zero. Flink's code path only uses
> aircompressor's LZO and ZSTD implementations, so the vulnerable decompressors
> are bundled but not invoked; the upgrade is dependency hygiene and removes
> security-scanner findings. The packages (io.airlift.compress.*), API and
> bytecode level (Java 8) are unchanged, making this a drop-in replacement.
> Why not aircompressor-v3: the actively developed line was renamed to the
> io.airlift:aircompressor-v3 artifact with packages under
> io.airlift.compress.v3. Its latest release (3.6) is compiled to Java 25
> bytecode (earlier 3.x releases required Java 22), while Flink still compiles
> at source level Java 11 and supports Java 11, 17 and 21 at runtime. The
> package rename would additionally require code changes. v3 is therefore not
> an option until Flink drops support for pre-22 JVMs.
> Alternatives considered:
> * Replacing the ZSTD codec with com.github.luben:zstd-jni (native libzstd via
> JNI, as used by Kafka/Spark/Parquet) would likely be faster, but swapping
> pure Java for native libraries in the shuffle hot path is a
> performance-motivated change that needs benchmarks and its own ticket.
> * The LZO codec has no alternative: native liblzo2 and hadoop-lzo are
> GPL-licensed, and aircompressor's LZO is the only Apache-2.0-licensed Java
> implementation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
