It’s time to send our project’s quarterly report to the ASF board on Wednesday November 8th. Here’s what I wrote as a draft; let me know any suggested changes.
============================= Issues for the board: - None Project status: - We released Apache Spark 3.5 on September 15, a feature release with over 1300 patches. This release introduced more scenarios with general availability for Spark Connect, like Scala and Go client, distributed training and inference support, and enhancement of compatibility for Structured streaming. It also introduced new PySpark and SQL functionality, including the SQL IDENTIFIER clause, named argument support for SQL function calls, SQL function support for HyperLogLog approximate aggregations, and Python user-defined table functions; simplified distributed training with DeepSpeed; introduced watermark propagation among operators; and added the dropDuplicatesWithinWatermark operation in Structured Streaming. - We made a patch release, Spark 3.3.3, on August 21, 2023. - Apache Spark 4.0.0-SNAPSHOT is now ready for Java 21. [SPARK-43831] - The vote on "Updating documentation hosted for EOL and maintenance releases" has passed. - The vote on the Spark Project Improvement Proposals (SPIPs) for "State Data Source - Reader" has passed. - The PMC has voted to add two new PMC members, Yuanjian Li and Yikun Jiang, and one new committer, Jiaan Geng, to the project. Trademarks: - No changes since the last report. Latest releases: - Spark 3.5.0 was released on September 13, 2023 - Spark 3.3.3 was released on August 21, 2023 - Spark 3.4.1 was released on June 23, 2023 Committers and PMC: - The latest committer was added on Oct 2nd, 2023 (Jiaan Geng). - The latest PMC members were added on Oct 2nd, 2023 (Yuanjian Li and Yikun Jiang). ============================= --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
