Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

Yeah let's get that fix in, but it seems to be a minor test only issue so should not block release. On Fri, Feb 16, 2024, 9:30 AM yangjie01 wrote: > Very sorry. When I was fixing `SPARK-45242 ( > https://github.com/apache/spark/pull/43594)` > , I

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

Very sorry. When I was fixing `SPARK-45242 (https://github.com/apache/spark/pull/43594)`, I noticed that its `Affects Version` and `Fix Version` of SPARK-45242 were both 4.0, and I didn't realize that it had also been merged into branch-3.5, so I didn't advocate for SPARK-45357 to be

Re: [VOTE] Release Apache Spark 3.5.1 (RC2)

I traced back relevant changes and got a sense of what happened. Yangjie figured out the issue via link . It's a tricky issue according to the comments from Yangjie - the test is dependent on ordering of execution for test suites.

Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow

Hi Chao, As a cool feature - Compared to standard Spark, what kind of performance gains can be expected with Comet? - Can one use Comet on k8s in conjunction with something like a Volcano addon? HTH Mich Talebzadeh, Dad | Technologist | Solutions Architect | Engineer London