Hi Zhentao, Thanks for initiating the discussion and for volunteering to serve as the release manager. +1 on starting the Celeborn 0.7.0 release. Given the substantial features, improvements, and fixes accumulated since 0.6.x, now is a good time to cut the release branch and begin the RC process.
I also support the proposal to target 1.0.0 as the next major release after 0.7.0. The project has developed a mature feature set and has seen broad production adoption. After 0.7.0, it would be helpful to align on explicit 1.0.0 readiness criteria, particularly around API and protocol compatibility, upgrade paths, documentation, integration coverage, and production validation. Thanks again for driving the release. Best regards, Nicholas Jiang On 2026/07/22 08:02:15 Zhentao Shuai wrote: > Hi Celeborn community, > I'd like to kick off the 0.7.0 release now that we're at the end of July. > It has been about 10 months since 0.6.0, and master has accumulated a > substantial set of new features, improvements, and fixes since 0.6.3 alone. > Highlights include: > ● Engine support. Spark 4.1/4.2 and Flink 2.1/2.2/2.3 support; Spark 4.0 and > Flink 2.0 client maturity. Removed Spark 2.4, Flink 1.16/1.17, and Hadoop 2 > client support. > ● C++ native client (CIP-14). Full push/merge data path, revive, LZ4/ZSTD > compression, retry logic, heartbeat decoding, native transport, and > C++-write/Java-read hybrid integration tests — a major push toward feature > parity with the Java client. > ● Encryption at Rest (CIP-22). Spark-side implementation landed. > ● Shuffle integrity. End-to-end integrity check for Flink, iterator > fully-consumed validation, and extended E2E checked zone. > ● Remote storage (S3/OSS/HDFS). S3 client caching, credentials provider > support, DFS replication factor configuration, MapPartitionData on DFS, and > multiple robustness fixes. > ● Observability. Flush latency per storage tier, eviction counts, read buffer > availability, metadata operation metrics, and more — all reflected in the > Grafana dashboards. > ● Performance. Zero-copy sendfile in Netty native transports, parallelized > open stream and commit paths, and reduced lock contention on hot paths. > ● Durability & reliability. fsync on commit, automatic RocksDB restoration, > disk-full rejection, Ratis stepdown and outbox retry, and numerous > race-condition and resource-leak fixes. > ● Security. RPC authorization, Java deserialization filter, and configurable > HTTP auth bypass paths. > ● Standalone LifecycleManager and Rust SDK, plus Helm chart improvements. > Looking ahead: 1.0.0 > With 0.7.0 we are essentially completing the feature surface for a 1.0. I'd > like to propose that the next major release after 0.7.0 will be 1.0.0 — > marking Celeborn's technical maturity and production-grade stability, > reflecting the collective work since graduation: Spark stage rerun, MapReduce > and Flink support, memory and remote storage, Flink hybrid shuffle, > HARD_SPLIT in PushMergedData, skew partition optimization, the C++ client, > encryption at rest, and broad real-world deployment feedback. > I'm volunteering to serve as the release manager for 0.7.0. If there are no > objections, I'll start preparing the release branch and the first RC after > this thread concludes (72 hours, per convention). > Regards, > Zhentao Shuai >
