I would like to discuss if we can introduce a change here - maybe it’s also
possible to get that into 2.4.4?
I’m keen to contribute that change. Please let me know your thoughts on this.
King Regards
Roland Johann
Software Developer/Data Engineer
phenetic GmbH
Lütticher Straße 10, 50674 Köln,
Unfortunately, I need to -1.
Recently we found that the repartition correctness bug can still be
reproduced. The root cause has been identified and there are 2 PRs to fix 2
related issues:
https://github.com/apache/spark/pull/25491
https://github.com/apache/spark/pull/25498
I think we should
Due to the recent correctness issue at SPARK-28699, I will delay the
release for Spark 2.3.4 RC1 for a while.
https://issues.apache.org/jira/browse/SPARK-28699?focusedCommentId=16910859=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16910859
Regards,
Kazuaki Ishizaki
Thank you for testing, Sean and Herman.
There are three reporting until now.
1. SPARK-28775 is for JDK 8u221+ testing at Apache Spark 3.0/2.4/2.3.
2. SPARK-28749 is for Scala 2.12 Python testing at Apache Spark 2.4 only.
3. SPARK-28699 is for disabling radix sort for ShuffleExchangeExec at
Things are looking pretty good so far, but a few notes:
I thought we might need this PR to make the 2.12 build of 2.4.x not
try to build Kafka 0.8 support, but, I'm not seeing that 2.4.x + 2.12
builds or tests it?
https://github.com/apache/spark/pull/25482
I can merge this to 2.4 shortly anyway,
Please vote on releasing the following candidate as Apache Spark version
2.4.4.
The vote is open until August 22nd 10AM PST and passes if a majority +1 PMC
votes are cast, with a minimum of 3 +1 votes.
[ ] +1 Release this package as Apache Spark 2.4.4
[ ] -1 Do not release this package because
