Hello All, I would like to bring on the table the theme about the new Spark release 3.3. According to the public schedule at https://spark.apache.org/versioning-policy.html, we planned to start the code freeze and release branch cut on March 15th, 2022. Since this date is coming soon, I would like to take your attention on the topic and gather objections that you might have.
Bellow is the list of ongoing and active SPIPs: Spark SQL: - [SPARK-31357] DataSourceV2: Catalog API for view metadata - [SPARK-35801] Row-level operations in Data Source V2 - [SPARK-37166] Storage Partitioned Join Spark Core: - [SPARK-20624] Add better handling for node shutdown - [SPARK-25299] Use remote storage for persisting shuffle data PySpark: - [SPARK-26413] RDD Arrow Support in Spark Core and PySpark Kubernetes: - [SPARK-36057] Support Customized Kubernetes Schedulers Probably, we should finish if there are any remaining works for Spark 3.3, and switch to QA mode, cut a branch and keep everything on track. I would like to volunteer to help drive this process. Best regards, Max Gekk
