FYI, three more patches landed to `branch-2.4` until yesterday.
[SPARK-28642][SQL][2.4] Hide credentials in show create table
[SPARK-27330][SS][2.4] support task abort in foreach writer
[SPARK-28025][SS][2.4] Fix FileContextBasedCheckpointFileManager leaking
crc files
For some non-blocker
Hi, All.
This 2.4.4 RC1 vote fails and the reported PRs are merged to `branch-2.4`.
The following is the commit list since `v2.4.4-rc1` tag.
Preparing development version 2.4.5-SNAPSHOT
[SPARK-28749][TEST][BRANCH-2.4] Fix PySpark tests not to require kafka-0-8
[SPARK-28775][CORE][TESTS] Skip
Sounds fine, we probably needed SPARK-28775 anyway. I merged that and
SPARK-28749. It looks like it's just the one you're talking about
right now, SPARK-28699.
The rest of the tests seemed to pass OK, release looks good, but bears
more testing by everyone out there before a next RC.
On Tue, Aug
Unfortunately, I need to -1.
Recently we found that the repartition correctness bug can still be
reproduced. The root cause has been identified and there are 2 PRs to fix 2
related issues:
https://github.com/apache/spark/pull/25491
https://github.com/apache/spark/pull/25498
I think we should
Thank you for testing, Sean and Herman.
There are three reporting until now.
1. SPARK-28775 is for JDK 8u221+ testing at Apache Spark 3.0/2.4/2.3.
2. SPARK-28749 is for Scala 2.12 Python testing at Apache Spark 2.4 only.
3. SPARK-28699 is for disabling radix sort for ShuffleExchangeExec at
Things are looking pretty good so far, but a few notes:
I thought we might need this PR to make the 2.12 build of 2.4.x not
try to build Kafka 0.8 support, but, I'm not seeing that 2.4.x + 2.12
builds or tests it?
https://github.com/apache/spark/pull/25482
I can merge this to 2.4 shortly anyway,
Please vote on releasing the following candidate as Apache Spark version
2.4.4.
The vote is open until August 22nd 10AM PST and passes if a majority +1 PMC
votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.4.4
[ ] -1 Do not release this package because
