Re: [VOTE] SPARK 2.4.0 (RC2)

> > Using Scala 2.12.7 is a not an infra change, but change to the build, > but again it's not even specific to 2.12.7. We should use the latest > if we can though. > > yep, exactly. we don't even have scala install on any of our jenkins nodes. this is all taken care of via build/mvn at runtime.

Re: [VOTE] SPARK 2.4.0 (RC2)

-1. Found an issue in a new 2.4 Java API: https://issues.apache.org/jira/browse/SPARK-25644 We should fix it in 2.4.0 to avoid future breaking changes. Best Regards, Ryan On Mon, Oct 1, 2018 at 7:22 PM Michael Heuer wrote: > FYI I’ve open two new issues against 2.4.0 rc2 > >

Re: [VOTE] SPARK 2.4.0 (RC2)

FYI I’ve open two new issues against 2.4.0 rc2 https://issues.apache.org/jira/browse/SPARK-25587 https://issues.apache.org/jira/browse/SPARK-25588 that are regressions against 2.3.1, and may

Re: [VOTE] SPARK 2.4.0 (RC2)

This RC fails because of the correctness bug: SPARK-25538 I'll start a new RC once the fix(https://github.com/apache/spark/pull/22602) is merged. Thanks, Wenchen On Tue, Oct 2, 2018 at 1:21 AM Sean Owen wrote: > Given that this release is probably still 2 weeks from landing, I don't > think

Re: [VOTE] SPARK 2.4.0 (RC2)

tt 2018 alle ore 09:11 Ted Yu ha >> scritto: >> >>> +1 >>> >>> Original message >>> From: Denny Lee >>> Date: 9/30/18 10:30 PM (GMT-08:00) >>> To: Stavros Kontopoulos >>> Cc: Sean Owen , Wenchen Fan , >&g

Re: [VOTE] SPARK 2.4.0 (RC2)

Original message >> From: Denny Lee >> Date: 9/30/18 10:30 PM (GMT-08:00) >> To: Stavros Kontopoulos >> Cc: Sean Owen , Wenchen Fan , dev >> >> Subject: Re: [VOTE] SPARK 2.4.0 (RC2) >> >> +1 (non-binding) >> >> >>

Re: [VOTE] SPARK 2.4.0 (RC2)

hen Fan , dev < > dev@spark.apache.org> > Subject: Re: [VOTE] SPARK 2.4.0 (RC2) > > +1 (non-binding) > > > On Sat, Sep 29, 2018 at 10:24 AM Stavros Kontopoulos < > stavros.kontopou...@lightbend.com> wrote: > >> +1 >> >> Stavros >

Re: [VOTE] SPARK 2.4.0 (RC2)

+1 Original message From: Denny Lee Date: 9/30/18 10:30 PM (GMT-08:00) To: Stavros Kontopoulos Cc: Sean Owen , Wenchen Fan , dev Subject: Re: [VOTE] SPARK 2.4.0 (RC2) +1 (non-binding) On Sat, Sep 29, 2018 at 10:24 AM Stavros Kontopoulos wrote: +1 Stavros On Sat, Sep

Re: [VOTE] SPARK 2.4.0 (RC2)

+1 (non-binding) On Sat, Sep 29, 2018 at 10:24 AM Stavros Kontopoulos < stavros.kontopou...@lightbend.com> wrote: > +1 > > Stavros > > On Sat, Sep 29, 2018 at 5:59 AM, Sean Owen wrote: > >> +1, with comments: >> >> There are 5 critical issues for 2.4, and no blockers: >> SPARK-25378

Re: [VOTE] SPARK 2.4.0 (RC2)

+1 Stavros On Sat, Sep 29, 2018 at 5:59 AM, Sean Owen wrote: > +1, with comments: > > There are 5 critical issues for 2.4, and no blockers: > SPARK-25378 ArrayData.toArray(StringType) assume UTF8String in 2.4 > SPARK-25325 ML, Graph 2.4 QA: Update user guide for new features & APIs >

Re: [VOTE] SPARK 2.4.0 (RC2)

+1, with comments: There are 5 critical issues for 2.4, and no blockers: SPARK-25378 ArrayData.toArray(StringType) assume UTF8String in 2.4 SPARK-25325 ML, Graph 2.4 QA: Update user guide for new features & APIs SPARK-25319 Spark MLlib, GraphX 2.4 QA umbrella SPARK-25326 ML, Graph 2.4 QA:

Re: [VOTE] SPARK 2.4.0 (RC2)

Go ahead and file a JIRA to update to 2.12.7 with these details. We'll assess whether it is a blocker. On Fri, Sep 28, 2018 at 12:09 PM Darcy Shen wrote: > > I agree it is a non-important Spark bug. I mean the Option and String > comparison. The bug is easy to fix and obvious to confirm. If the

Re: [VOTE] SPARK 2.4.0 (RC2)

I agree it is a non-important Spark bug. I mean the Option and String comparison. The bug is easy to fix and obvious to confirm. If the desc of PR may not be accurate，feel free to edit the title or content. I am on a vocation from 9.29   ：）But the scala bug of

Re: [VOTE] SPARK 2.4.0 (RC2)

Hi, Wenchen. The current issue link seems to be out of order for me. The list of bug fixes going into 2.4.0 can be found at the following URL: https://issues.apache.org/jira/projects/SPARK/versions/2.4.0 Could you send out with the following issue link for next RCs?

Re: [VOTE] SPARK 2.4.0 (RC2)

I don't even know how or if this manifests as a bug. The code is indeed incorrect and the 2.12 compiler flags it. We fixed a number of these in SPARK-25398. While I want to get this into 2.4 if we have another RC, I don't see evidence this is a blocker. It is not specific to Scala 2.12. Using

Re: [VOTE] SPARK 2.4.0 (RC2)

I don't think this bug is so serious to fail an RC, it's only about metrics IIUC, and it's not a regression in 2.4. I agree we should backport this fix to 2.3 and 2.4, and we should update our scala 2.12 jenkins build to use scala 2.12.7. cc Shane do you know how to change it? BTW end users can

Re: [VOTE] SPARK 2.4.0 (RC2)

Yes, that's proposed by Sean. This time we should publish a Scala 2.12 build, both in maven and the download page. On Fri, Sep 28, 2018 at 11:34 AM Saisai Shao wrote: > Only "without-hadoop" profile has 2.12 binary, is it expected? > > Thanks > Saisai > > Wenchen Fan 于2018年9月28日周五 上午11:08写道： >

Re: [VOTE] SPARK 2.4.0 (RC2)

Only "without-hadoop" profile has 2.12 binary, is it expected? Thanks Saisai Wenchen Fan 于2018年9月28日周五 上午11:08写道： > I'm adding my own +1, since all the problems mentioned in the RC1 voting > email are all resolved. And there is no blocker issue for 2.4.0 AFAIK. > > On Fri, Sep 28, 2018 at

Re: [VOTE] SPARK 2.4.0 (RC2)

I'm adding my own +1, since all the problems mentioned in the RC1 voting email are all resolved. And there is no blocker issue for 2.4.0 AFAIK. On Fri, Sep 28, 2018 at 10:59 AM Wenchen Fan wrote: > Please vote on releasing the following candidate as Apache Spark version > 2.4.0. > > The vote is

[VOTE] SPARK 2.4.0 (RC2)

Please vote on releasing the following candidate as Apache Spark version 2.4.0. The vote is open until October 1 PST and passes if a majority +1 PMC votes are cast, with a minimum of 3 +1 votes. [ ] +1 Release this package as Apache Spark 2.4.0 [ ] -1 Do not release this package because ... To