Hi Nicholas, Thanks for driving this. This proposal can effectively reduce future maintenance workload and prevent necessary refactorings from being constrained by legacy version compatibility. Overall +1.
According to the Flink official website (link <https://flink.apache.org/downloads/#update-policy-for-old-releases>), Flink maintains the two most recent minor versions, and 1.20 as an LTS. I believe it is reasonable to gradually deprecate versions that have already reached their end of maintenance upstream. Considering production practices, it makes sense to allow an additional grace period of about a year before complete removal. Given that Flink 1.18.1 was released in January 2024 and its security support ended in March 2025, it has been EOL for over a year now. As the most recent EOL'd version, using 1.18 as the boundary aligns with the one-year grace period principle. Therefore, I think this division is fine. Removal of 1.16/1.17 may happen two minor versions later, as planned it would be 1.0. Additionally, the actual removal of Flink 1.16 and 1.17 support could be deferred until two minor releases from now. Based on the current roadmap, this timeline perfectly aligns with the upcoming major release 1.0. Best, Gen On Mon, Jun 15, 2026 at 2:34 PM Nicholas <[email protected]> wrote: > Hi Celeborn community, > > > I'd like to start a discussion about reducing the number of Spark and > Flink versions we support. > > > # Background > > > Celeborn currently supports a very wide build matrix: > > > - Spark: 2.4, 3.0, 3.1, 3.2, 3.3, 3.4, 3.5, 4.0, 4.1 > (9 profiles; Spark 2.4 also forces us to keep a Scala 2.11 build > and a separate client-spark/spark-2 module) > - Flink: 1.16, 1.17, 1.18, 1.19, 1.20, 2.0, 2.1, 2.2 > (8 profiles, each with its own *-shaded module) > > > This breadth has a real cost: > > > - CI time and flakiness grow with every profile we build and test. > - The release process has to cross-build and stage artifacts for the full > matrix, which slows down every release. > - Many of these engines are already end-of-life upstream and receive no > further releases, so we are maintaining and shipping clients for versions > their own communities no longer support. > > > # Proposal > > > Keep the most recent, actively-maintained versions and deprecate + remove > the oldest ones. As a concrete starting point: > > > Spark > - Deprecate now, remove in <next major/minor>: 2.4, 3.0, 3.1 > - Keep: 3.2, 3.3, 3.4, 3.5, 4.0, 4.1 > - Dropping 2.4 lets us also drop the Scala 2.11 build and the dedicated > spark-2 module. > > > Flink > - Deprecate now, remove in <next major/minor>: 1.16, 1.17 > - Keep: 1.18, 1.19, 1.20, 2.0, 2.1, 2.2 > > > # Suggested process > > > 1. Mark the above versions as deprecated in the next release: a note in > the docs/release notes, and optionally a startup log warning. > 2. Remove the corresponding Maven profiles and modules in the release > after that. > 3. Keep the existing release branches as-is, so users who still run these > engines can continue to use the last Celeborn release that supports them. > > > # Open questions > > > - Is the proposed cut line right, or should we be more / less aggressive > (e.g. also drop Spark 3.2 / Flink 1.18)? > - Are there users still relying on any of the versions proposed for > removal? Please speak up here so we can account for that. > - Which target release should carry the deprecation vs. the removal? > > > Looking forward to your feedback. > > > Thanks, > Nicholas Jiang
