Re: [VOTE] [SPIP] SPARK-15689: Data Source API V2 read path

+1 (non-binding) I have found the suggestion from Andrew Ash and James about plan push down quite interesting. However, I am not clear about the join push-down support at the data source level. Shouldn't it be the responsibility of the join node to carry out a data source specific join? I mean

Supporting Apache Aurora as a cluster manager

Hi Spark Devs, We are using Aurora (http://aurora.apache.org/) as our mesos framework for running stateless services. We would like to use Aurora to deploy big data and batch workloads as well. And for this we have forked Spark and implement the ExternalClusterManager trait. The reason for doing

Re: [VOTE] [SPIP] SPARK-15689: Data Source API V2 read path

+1 Regards, Vaquar khan On Sep 10, 2017 5:18 AM, "Noman Khan" wrote: > +1 > -- > *From:* wangzhenhua (G) > *Sent:* Friday, September 8, 2017 2:20:07 AM > *To:* Dongjoon Hyun; 蒋星博 > *Cc:* Michael Armbrust; Reynold Xin;

Re: [SS] Bug in StreamExecution? currentBatchId and getBatchDescriptionString for web UI

Hi, Please disregard my finding. It does not seem a bug, but just a small "dead code" as "init" will never be displayed in web UI = the minimum batch id can ever be 0 and so getBatchDescriptionString could be a little "improved". Sorry for the noise. Pozdrawiam, Jacek Laskowski

Re: [VOTE] [SPIP] SPARK-15689: Data Source API V2 read path

+1 From: wangzhenhua (G) Sent: Friday, September 8, 2017 2:20:07 AM To: Dongjoon Hyun; 蒋星博 Cc: Michael Armbrust; Reynold Xin; Andrew Ash; Herman van Hövell tot Westerflier; Ryan Blue; Spark dev list; Suresh Thalamati; Wenchen Fan Subject: