[RESULT] [VOTE][SPARK-30602] SPIP: Support push-based shuffle to improve shuffle efficiency

Hi, The vote passed with 16 +1's (6 binding) and no -1's +1s (* = binding): Xingbo Jiang Venkatakrishnan Sowrirajan Tom Graves (*) Chandni Singh DB Tsai (*) Xiao Li (*) Angers Zhu Joseph Torres Kalyan Dongjoon Hyun (*) Wenchen Fan (*) Yi Wu 叶先进 郑瑞峰 Takeshi Yamamuro Mridul Muralidharan (*)

Re: [VOTE][SPARK-30602] SPIP: Support push-based shuffle to improve shuffle efficiency

Adding my +1 as well, before closing the vote. Regards, Mridul On Sun, Sep 13, 2020 at 9:59 PM Mridul Muralidharan wrote: > Hi, > > I'd like to call for a vote on SPARK-30602 - SPIP: Support push-based > shuffle to improve shuffle efficiency. > Please take a look at: > >- SPIP jira:

Inconsistencies with how catalyst optimizer handles non-deterministic expressions

Hello, I believe, that currently non-deterministic expressions are handled in two conflicting approaches in the catalyst optimizer. The first approach is the one I have seen in the recent pull request reviews - the optimizer should never change the number of times a non-deterministic expression

RE: Spark-Locality: Hinting Spark location of the executor does not take effect.

Was providing IP address instead of FQDN. Providing FQDN helped. Thanks, From: Nasrulla Khan Haris Sent: Wednesday, September 16, 2020 4:11 PM To: dev@spark.apache.org Subject: Spark-Locality: Hinting Spark location of the executor does not take effect. HI Spark developers, If I want to hint

Pre query execution hook for custom datasources

Hi, In our custom datasource implementation, we want to inject some query level information. For example - scala> val df = spark.sql("some query") // uses custom datasource under the hood through Session Extensions. scala> df.count // here we want some kind of pre execution hook just before