Re: [VOTE] [SPARK-24374] SPIP: Support Barrier Scheduling in Apache Spark

2018-06-04 Thread Xiangrui Meng
+1 from myself. The vote passed with the following +1s: * Susham kumar reddy Yerabolu * Xingbo Jiang * Xiao Li* * Weichen Xu * Joseph Bradley* * Henry Robinson * Xiangrui Meng* * Wenchen Fan* Henry, you can find a design sketch at https://issues.apache.org/jira/browse/SPARK-24375. To help

Re: TextSocketMicroBatchReader no longer supports nc utility

2018-06-04 Thread Jungtaek Lim
Yeah that's why I initiated this thread, especially socket source is expected to be used from examples on official document or some experiments, which we tend to simply use netcat. I'll file an issue and provide the fix. 2018년 6월 5일 (화) 오전 1:48, Joseph Torres 님이 작성: > I tend to agree that this

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread Xiao Li
+1 On Mon, Jun 4, 2018 at 12:44 PM Henry Robinson wrote: > +1 (non-binding) > > On 4 June 2018 at 11:15, Bryan Cutler wrote: > >> +1 >> >> On Mon, Jun 4, 2018 at 10:18 AM, Joseph Bradley >> wrote: >> >>> +1 >>> >>> On Mon, Jun 4, 2018 at 10:16 AM, Mark Hamstra >>> wrote: >>> +1

[SS] Invalid call to dataType on unresolved object

2018-06-04 Thread Lalwani, Jayesh
This is possibly a bug introduced in 2.3.0. I have this code that I run in spark-shell spark.readStream.format("socket").option("host", "localhost").option("port", ).load().toDF("employeeId"). | withColumn("swipeTime", expr("current_timestamp()")). |

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread Henry Robinson
+1 (non-binding) On 4 June 2018 at 11:15, Bryan Cutler wrote: > +1 > > On Mon, Jun 4, 2018 at 10:18 AM, Joseph Bradley > wrote: > >> +1 >> >> On Mon, Jun 4, 2018 at 10:16 AM, Mark Hamstra >> wrote: >> >>> +1 >>> >>> On Fri, Jun 1, 2018 at 3:29 PM Marcelo Vanzin >>> wrote: >>> Please

Re: [VOTE] [SPARK-24374] SPIP: Support Barrier Scheduling in Apache Spark

2018-06-04 Thread Wenchen Fan
+1 On Tue, Jun 5, 2018 at 1:20 AM, Henry Robinson wrote: > +1 > > (I hope there will be a fuller design document to review, since the SPIP > is really light on details). > > On 4 June 2018 at 10:17, Joseph Bradley wrote: > >> +1 >> >> On Sun, Jun 3, 2018 at 9:59 AM, Weichen Xu >> wrote: >>

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread Bryan Cutler
+1 On Mon, Jun 4, 2018 at 10:18 AM, Joseph Bradley wrote: > +1 > > On Mon, Jun 4, 2018 at 10:16 AM, Mark Hamstra > wrote: > >> +1 >> >> On Fri, Jun 1, 2018 at 3:29 PM Marcelo Vanzin >> wrote: >> >>> Please vote on releasing the following candidate as Apache Spark version >>> 2.3.1. >>> >>>

Re: [VOTE] [SPARK-24374] SPIP: Support Barrier Scheduling in Apache Spark

2018-06-04 Thread Henry Robinson
+1 (I hope there will be a fuller design document to review, since the SPIP is really light on details). On 4 June 2018 at 10:17, Joseph Bradley wrote: > +1 > > On Sun, Jun 3, 2018 at 9:59 AM, Weichen Xu > wrote: > >> +1 >> >> On Fri, Jun 1, 2018 at 3:41 PM, Xiao Li wrote: >> >>> +1 >>> >>>

Unsubscribe

2018-06-04 Thread Al Pivonka
On Mon, Jun 4, 2018, 1:18 PM Joseph Bradley wrote: > +1 > > On Mon, Jun 4, 2018 at 10:16 AM, Mark Hamstra > wrote: > >> +1 >> >> On Fri, Jun 1, 2018 at 3:29 PM Marcelo Vanzin >> wrote: >> >>> Please vote on releasing the following candidate as Apache Spark version >>> 2.3.1. >>> >>> Given that

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread Joseph Bradley
+1 On Mon, Jun 4, 2018 at 10:16 AM, Mark Hamstra wrote: > +1 > > On Fri, Jun 1, 2018 at 3:29 PM Marcelo Vanzin wrote: > >> Please vote on releasing the following candidate as Apache Spark version >> 2.3.1. >> >> Given that I expect at least a few people to be busy with Spark Summit >> next >>

Re: [VOTE] [SPARK-24374] SPIP: Support Barrier Scheduling in Apache Spark

2018-06-04 Thread Joseph Bradley
+1 On Sun, Jun 3, 2018 at 9:59 AM, Weichen Xu wrote: > +1 > > On Fri, Jun 1, 2018 at 3:41 PM, Xiao Li wrote: > >> +1 >> >> 2018-06-01 15:41 GMT-07:00 Xingbo Jiang : >> >>> +1 >>> >>> 2018-06-01 9:21 GMT-07:00 Xiangrui Meng : >>> Hi all, I want to call for a vote of SPARK-24374

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread Mark Hamstra
+1 On Fri, Jun 1, 2018 at 3:29 PM Marcelo Vanzin wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.3.1. > > Given that I expect at least a few people to be busy with Spark Summit next > week, I'm taking the liberty of setting an extended voting period. The

Re: TextSocketMicroBatchReader no longer supports nc utility

2018-06-04 Thread Joseph Torres
I tend to agree that this is a bug. It's kinda silly that nc does this, but a socket connector that doesn't work with netcat will surely seem broken to users. It wouldn't be a huge change to defer opening the socket until a read is actually required. On Sun, Jun 3, 2018 at 9:55 PM, Jungtaek Lim

Re: [VOTE] Spark 2.3.1 (RC4)

2018-06-04 Thread John Zhuge
+1 On Sun, Jun 3, 2018 at 6:12 PM, Hyukjin Kwon wrote: > +1 > > 2018년 6월 3일 (일) 오후 9:25, Ricardo Almeida 님이 > 작성: > >> +1 (non-binding) >> >> On 3 June 2018 at 09:23, Dongjoon Hyun wrote: >> >>> +1 >>> >>> Bests, >>> Dongjoon. >>> >>> On Sat, Jun 2, 2018 at 8:09 PM, Denny Lee wrote: >>>