It’s time for our quarterly report to the ASF board, which we need to submit on Wednesday. I’ve put together the following draft based on activity in the community — let me know if you’d like to add or change anything:
====================================== Description: Apache Spark is a fast and general engine for large-scale data processing. It offers high-level APIs in Java, Scala, Python, R and SQL as well as a rich set of libraries including stream processing, machine learning, and graph analytics. Issues for the board: - None Project status: - We released Apache Spark 3.1.1, a major update release for the 3.x branch, on March 2nd. This release includes updates to improve Python usability and error messages, ANSI SQL support, the streaming UI, and support for running Apache Spark on Kubernetes, which is now marked GA. Overall, the release includes about 1500 patches. - We are voting on an Apache Spark 2.4.8 bug fix release with for the Spark 2.x line. This may be the last release on 2.x. - We added six new committers to the project: Atilla Zsolt Piros, Gabor Somogyi, Kent Yao, Maciej Szymkiewicz, Max Gekk, and Yi Wu. - Several SPIPs (major project improvement proposals) were voted on and accepted, including adding a Function Catalog in Spark SQL and adding a Pandas API layer for PySpark based on the Koalas project. We’ve also started an effort to standardize error message reporting in Apache Spark (https://spark.apache.org/error-message-guidelines.html) so that messages are easier to understand and users can quickly figure out how to fix them. Trademarks: - No changes since the last report. Latest releases: - Spark 3.1.1 was released on March 2nd, 2021. - Spark 3.0.2 was released on February 19th, 2021. - Spark 2.4.7 was released on September 12th, 2020. Committers and PMC: - The latest committers were added on March 11th, 2021 (Atilla Zsolt Piros, Gabor Somogyi, Kent Yao, Maciej Szymkiewicz, Max Gekk, and Yi Wu). - The latest PMC member was added on Sept 4th, 2019 (Dongjoon Hyun). The PMC has been discussing some new PMC candidates. --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
