Unsubscribe

Unsubscribe This message contains information that may be privileged or confidential and is the property of the Capgemini Group. It is intended only for the person to whom it is addressed. If you are not the intended recipient, you are not authorized to read, print, retain, copy, disseminate,

Re: Time for 2.3.2?

+1, like mentioned by Marcelo, these issues seems quite severe. I can work on the release if short of hands :). Thanks Jerry Marcelo Vanzin 于2018年6月28日周四 上午11:40写道： > +1. SPARK-24589 / SPARK-24552 are kinda nasty and we should get fixes > for those out. > > (Those are what delayed 2.2.2 and

Re: Time for 2.3.2?

+1. SPARK-24589 / SPARK-24552 are kinda nasty and we should get fixes for those out. (Those are what delayed 2.2.2 and 2.1.3 for those watching...) On Wed, Jun 27, 2018 at 7:59 PM, Wenchen Fan wrote: > Hi all, > > Spark 2.3.1 was released just a while ago, but unfortunately we discovered > and

Time for 2.3.2?

Hi all, Spark 2.3.1 was released just a while ago, but unfortunately we discovered and fixed some critical issues afterward. *SPARK-24495: SortMergeJoin may produce wrong result.* This is a serious correctness bug, and is easy to hit: have duplicated join key from the left table, e.g. `WHERE

Re: [VOTE] Spark 2.2.2 (RC2)

+1 On Thu, Jun 28, 2018 at 10:19 AM zhenya Sun wrote: > +1 > > 在 2018年6月28日，上午10:15，Hyukjin Kwon 写道： > > +1 > > 2018년 6월 28일 (목) 오전 8:42, Sean Owen 님이 작성: > >> +1 from me too. >> >> On Wed, Jun 27, 2018 at 3:31 PM Tom Graves >> wrote: >> >>> Please vote on releasing the following candidate as

Re: [VOTE] Spark 2.2.2 (RC2)

+1 > 在 2018年6月28日，上午10:15，Hyukjin Kwon 写道： > > +1 > > 2018년 6월 28일 (목) 오전 8:42, Sean Owen >님이 작성: > +1 from me too. > > On Wed, Jun 27, 2018 at 3:31 PM Tom Graves > wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.2.2. > > The

Re: [VOTE] Spark 2.2.2 (RC2)

+1 2018년 6월 28일 (목) 오전 8:42, Sean Owen 님이 작성: > +1 from me too. > > On Wed, Jun 27, 2018 at 3:31 PM Tom Graves > wrote: > >> Please vote on releasing the following candidate as Apache Spark version >> 2.2.2. >> >> The vote is open until Mon, July 2nd @ 9PM UTC (2PM PDT) and passes if a >>

Re: [VOTE] Spark 2.1.3 (RC2)

On Wed, Jun 27, 2018 at 6:57 PM, Felix Cheung wrote: > Yes, this is broken with newer version of R. > > We check explicitly for warning for the R check which should fail the test > run. Hmm, something is missing somewhere then, because Jenkins seems mostly happy aside from a few flakes:

Re: [VOTE] Spark 2.1.3 (RC2)

Yes, this is broken with newer version of R. We check explicitly for warning for the R check which should fail the test run. From: Marcelo Vanzin Sent: Wednesday, June 27, 2018 6:55 PM To: Felix Cheung Cc: Marcelo Vanzin; Tom Graves; dev Subject: Re: [VOTE]

Re: [VOTE] Spark 2.1.3 (RC2)

Not sure I understand that bug. Is it a compatibility issue with new versions of R? It's at least marked as fixed in 2.2(.1). We do run jenkins on these branches, but that seems like just a warning, which would not fail those builds... On Wed, Jun 27, 2018 at 6:12 PM, Felix Cheung wrote: > (I

Re: [VOTE] Spark 2.1.3 (RC2)

(I don’t want to block the release(s) per se...) We need to backport SPARK-22281 (to branch-2.1 and branch-2.2) This is fixed in 2.3 back in Nov 2017 https://github.com/apache/spark/commit/2ca5aae47a25dc6bc9e333fb592025ff14824501#diff-e1e1d3d40573127e9ee0480caf1283d6 Perhaps we don't get

Re: [VOTE] Spark 2.2.2 (RC2)

+1 from me too. On Wed, Jun 27, 2018 at 3:31 PM Tom Graves wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.2.2. > > The vote is open until Mon, July 2nd @ 9PM UTC (2PM PDT) and passes if a > majority +1 PMC votes are cast, with a minimum of 3 +1 votes. > >

Re: [VOTE] Spark 2.2.2 (RC2)

+1 Checked sigs + ran a bunch of tests on the hadoop-2.7 binary package. On Wed, Jun 27, 2018 at 1:30 PM, Tom Graves wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.2.2. > > The vote is open until Mon, July 2nd @ 9PM UTC (2PM PDT) and passes if a > majority

[VOTE] Spark 2.2.2 (RC2)

Please vote on releasing the following candidate as Apache Spark version 2.2.2. The vote is open until Mon, July 2nd @ 9PM UTC (2PM PDT) and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.2.2 [ ] -1 Do not release this

Re: Unsubscribe

Unsubscribe On Fri, Jun 22, 2018 at 4:33 PM, Tarun Kumar wrote: > Unsubscribe

Re: Live Streamed Code Review today at 11am Pacific

Today @ 1:30pm pacific I'll be looking at the current Spark 2.1.3 RC and see how we validate Spark releases - https://www.twitch.tv/events/VAg-5PKURQeH15UAawhBtw / https://www.youtube.com/watch?v=1_XLrlKS26o . Tomorrow @ 12:30 live PR reviews & Monday live coding -

Re: Support SqlStreaming in spark

Structured Streaming supports standard SQL as the batch queries, so the users can switch their queries between batch and streaming easily. Could you clarify what problems SqlStreaming solves and what are the benefits of the new syntax? Best Regards, Ryan On Thu, Jun 14, 2018 at 7:06 PM, JackyLee

DatasourceV2 reader for binary files

Is anyone working on porting existing readers to DataSourcev2. Specifically, has anyone implemented a Datasource v2 reader for binary files? The information contained in this e-mail is confidential and/or proprietary to Capital One and/or

Re: [VOTE] Spark 2.1.3 (RC2)

+1 from me too for the usual reasons. On Tue, Jun 26, 2018 at 3:25 PM Marcelo Vanzin wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.1.3. > > The vote is open until Fri, June 29th @ 9PM UTC (2PM PDT) and passes if a > majority +1 PMC votes are cast, with a

Can we let tasks of broadcast job not wait for locality?

Hi All, I noticed the task scheduling will have a locality wait (default is 3s), which causes some tasks launched after a long delay(sometimes more than 3s), especially there are lots of tasks requesting to run concurrently and waiting for resources. Why not let tasks of broadcast job not to