Re: 3.3.1 Release

2022-10-25 Thread Dongjoon Hyun
It's released Today, Pastrana. https://downloads.apache.org/spark/spark-3.3.1/ https://spark.apache.org/news/spark-3-3-1-released.html https://spark.apache.org/releases/spark-release-3-3-1.html https://spark.apache.org/docs/3.3.1/ https://pypi.org/project/pyspark/3.3.1/ I guess the release

3.3.1 Release

2022-10-25 Thread Pastrana, Rodrigo (RIS-BCT)
Thanks to all involved with the 3.3.1 release. Is there a target date for the official release? Thanks! [VOTE][RESULT] Release Spark 3.3.1 (RC4) The vote passes with 11 +1s (6 binding +1s). Thanks to all who helped with the release! (* = binding) +1: - Sean Owen (*) - Yang,Jie - Dongjoon Hyun

Re: Missing data in spark output

2022-10-25 Thread Steve Loughran
v1 on gcs isn't safe either as promotion from task attempt to successful task is a dir rename; fast and atomic on hdfs, O(files) and nonatomic on GCS. if i can get that hadoop 3.3.5 rc out soon, the manifest committer will be there to test https://issues.apache.org/jira/browse/MAPREDUCE-7341