Re: [VOTE] SPARK 2.3.2 (RC3)

2018-08-06 Thread Saisai Shao
Yes, there'll be an RC4, still waiting for the fix of one issue. Yuval Itzchakov 于2018年8月6日周一 下午6:10写道: > Are there any plans to create an RC4? There's an important Kafka Source > leak > fix I've merged back to the 2.3 branch. > > > > -- > Sent from:

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-08-06 Thread Yuval Itzchakov
Are there any plans to create an RC4? There's an important Kafka Source leak fix I've merged back to the 2.3 branch. -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail:

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-30 Thread Wenchen Fan
Another two correctness bug fixes were merged to 2.3 today: https://issues.apache.org/jira/browse/SPARK-24934 https://issues.apache.org/jira/browse/SPARK-24957 On Mon, Jul 30, 2018 at 1:19 PM Xiao Li wrote: > Sounds good to me. Thanks! Today, we merged another correctness fix >

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-29 Thread Xiao Li
Sounds good to me. Thanks! Today, we merged another correctness fix https://github.com/apache/spark/pull/21772. Xiao 2018-07-29 18:31 GMT-07:00 Saisai Shao : > Sure, I will do a next RC. I'm still waiting for a CVE fix, if this can be > done in this two days, I will also include that one. > >

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-29 Thread Saisai Shao
Sure, I will do a next RC. I'm still waiting for a CVE fix, if this can be done in this two days, I will also include that one. Xiao Li 于2018年7月28日周六 上午12:05写道: > The following blocker/important fixes have been merged to Spark 2.3 branch: > > https://issues.apache.org/jira/browse/SPARK-24927 >

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-27 Thread Xiao Li
The following blocker/important fixes have been merged to Spark 2.3 branch: https://issues.apache.org/jira/browse/SPARK-24927 https://issues.apache.org/jira/browse/SPARK-24867 https://issues.apache.org/jira/browse/SPARK-24891 *Saisai*, could you start the next RC? Thanks, Xiao 2018-07-20

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-20 Thread Tom Graves
fyi, I merged in a couple jira that were critical (and I thought would be good to include in the next release) that if we spin another RC will get included, we should update the jira SPARK-24755 and SPARK-24677, if anyone disagrees we could back those out but I think they would be good to

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Saisai Shao
Sure, I can wait for this and create another RC then. Thanks, Saisai Xiao Li 于2018年7月20日周五 上午9:11写道: > Yes. https://issues.apache.org/jira/browse/SPARK-24867 is the one I > created. The PR has been created. Since this is not rare, let us merge it > to 2.3.2? > > Reynold' PR is to get rid of

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Xiao Li
Yes. https://issues.apache.org/jira/browse/SPARK-24867 is the one I created. The PR has been created. Since this is not rare, let us merge it to 2.3.2? Reynold' PR is to get rid of AnalysisBarrier. That is better than multiple patches we added for AnalysisBarrier after 2.3.0 release. We can

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Saisai Shao
I see, thanks Reynold. Reynold Xin 于2018年7月20日周五 上午8:46写道: > Looking at the list of pull requests it looks like this is the ticket: > https://issues.apache.org/jira/browse/SPARK-24867 > > > > On Thu, Jul 19, 2018 at 5:25 PM Reynold Xin wrote: > >> I don't think my ticket should block this

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Reynold Xin
Looking at the list of pull requests it looks like this is the ticket: https://issues.apache.org/jira/browse/SPARK-24867 On Thu, Jul 19, 2018 at 5:25 PM Reynold Xin wrote: > I don't think my ticket should block this release. It's a big general > refactoring. > > Xiao do you have a ticket for

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Reynold Xin
I don't think my ticket should block this release. It's a big general refactoring. Xiao do you have a ticket for the bug you found? On Thu, Jul 19, 2018 at 5:24 PM Saisai Shao wrote: > Hi Xiao, > > Are you referring to this JIRA ( > https://issues.apache.org/jira/browse/SPARK-24865)? > > Xiao

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Saisai Shao
Hi Xiao, Are you referring to this JIRA ( https://issues.apache.org/jira/browse/SPARK-24865)? Xiao Li 于2018年7月20日周五 上午2:41写道: > dfWithUDF.cache() > dfWithUDF.write.saveAsTable("t") > dfWithUDF.write.saveAsTable("t1") > > > Cached data is not being used. It causes a big performance regression.

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Xiao Li
dfWithUDF.cache() dfWithUDF.write.saveAsTable("t") dfWithUDF.write.saveAsTable("t1") Cached data is not being used. It causes a big performance regression. 2018-07-19 11:32 GMT-07:00 Sean Owen : > What regression are you referring to here? A -1 vote really needs a > rationale. > > On Thu,

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Sean Owen
What regression are you referring to here? A -1 vote really needs a rationale. On Thu, Jul 19, 2018 at 1:27 PM Xiao Li wrote: > I would first vote -1. > > I might find another regression caused by the analysis barrier. Will keep > you posted. > >

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-19 Thread Xiao Li
I would first vote -1. I might find another regression caused by the analysis barrier. Will keep you posted. Xiao 2018-07-18 18:05 GMT-07:00 Takeshi Yamamuro : > +1 (non-binding) > > I run tests on a EC2 m4.2xlarge instance; > [ec2-user]$ java -version > openjdk version "1.8.0_171" > OpenJDK

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-18 Thread Takeshi Yamamuro
+1 (non-binding) I run tests on a EC2 m4.2xlarge instance; [ec2-user]$ java -version openjdk version "1.8.0_171" OpenJDK Runtime Environment (build 1.8.0_171-b10) OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode) On Thu, Jul 19, 2018 at 5:29 AM Ryan Blue wrote: > +1 (non-binding) > > On

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-18 Thread Ryan Blue
+1 (non-binding) On Wed, Jul 18, 2018 at 10:38 AM Denny Lee wrote: > +1 (non-binding) > On Tue, Jul 17, 2018 at 23:04 John Zhuge wrote: > >> +1 (non-binding) >> >> On Mon, Jul 16, 2018 at 8:04 PM Saisai Shao >> wrote: >> >>> I will put my +1 on this RC. >>> >>> For the test failure fix, I

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-18 Thread Denny Lee
+1 (non-binding) On Tue, Jul 17, 2018 at 23:04 John Zhuge wrote: > +1 (non-binding) > > On Mon, Jul 16, 2018 at 8:04 PM Saisai Shao > wrote: > >> I will put my +1 on this RC. >> >> For the test failure fix, I will include it if there's another RC. >> >> Sean Owen 于2018年7月16日周一 下午10:47写道: >> >

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-18 Thread John Zhuge
+1 (non-binding) On Mon, Jul 16, 2018 at 8:04 PM Saisai Shao wrote: > I will put my +1 on this RC. > > For the test failure fix, I will include it if there's another RC. > > Sean Owen 于2018年7月16日周一 下午10:47写道: > >> OK, hm, will try to get to the bottom of it. But if others can build this >>

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-16 Thread Saisai Shao
I will put my +1 on this RC. For the test failure fix, I will include it if there's another RC. Sean Owen 于2018年7月16日周一 下午10:47写道: > OK, hm, will try to get to the bottom of it. But if others can build this > module successfully, I give a +1 . The test failure is inevitable here and > should

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-16 Thread Sean Owen
OK, hm, will try to get to the bottom of it. But if others can build this module successfully, I give a +1 . The test failure is inevitable here and should not block release. On Sun, Jul 15, 2018 at 9:39 PM Saisai Shao wrote: > Hi Sean, > > I just did a clean build with mvn/sbt on 2.3.2, I

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-15 Thread Saisai Shao
Hi Sean, I just did a clean build with mvn/sbt on 2.3.2, I didn't meet the errors you pasted here. I'm not sure how it happens. Sean Owen 于2018年7月16日周一 上午6:30写道: > Looks good to me, with the following caveats. > > First see the discussion on > https://issues.apache.org/jira/browse/SPARK-24813

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-15 Thread Sean Owen
Looks good to me, with the following caveats. First see the discussion on https://issues.apache.org/jira/browse/SPARK-24813 ; the flaky HiveExternalCatalogVersionsSuite will probably fail all the time right now. That's not a regression and is a test-only issue, so don't think it must block the

Re: [VOTE] SPARK 2.3.2 (RC3)

2018-07-15 Thread Wenchen Fan
+1. The Spark 2.3 regressions I'm aware of are all fixed. On Sun, Jul 15, 2018 at 4:09 PM Saisai Shao wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.3.2. > > The vote is open until July 20 PST and passes if a majority +1 PMC votes > are cast, with a

[VOTE] SPARK 2.3.2 (RC3)

2018-07-15 Thread Saisai Shao
Please vote on releasing the following candidate as Apache Spark version 2.3.2. The vote is open until July 20 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.3.2 [ ] -1 Do not release this package because ... To