Re: [VOTE] Apache Spark 2.1.0 (RC5)

The vote passed with the following +1 and -1: +1 Reynold Xin* Sean Owen* Dongjoon Hyun Xiao Li Herman van Hövell tot Westerflier Joseph Bradley* Liwei Lin Denny Lee Holden Karau Adam Roberts vaquar khan 0/+1 (not sure what this means but putting it here just in case) Felix Cheung -1 Franklyn

Re: [VOTE] Apache Spark 2.1.0 (RC5)

Since it’s not a regression from 2.0 (I believe the same issue affects both 2.0 and 2.1) it doesn’t merit a -1 vote according to the voting guidelines. Of course, it would be nice if we could fix the various optimizer issues that all seem to have a workaround that involves persist() (another one

Re: [VOTE] Apache Spark 2.1.0 (RC5)

-1 https://issues.apache.org/jira/browse/SPARK-18589 hasn't been resolved by this release and is a blocker in our adoption of spark 2.0. I've updated the issue with some steps to reproduce the error. On Mon, Dec 19, 2016 at 4:37 AM, Sean Owen wrote: > PS, here are the open

Re: [VOTE] Apache Spark 2.1.0 (RC5)

PS, here are the open issues for 2.1.0. Forgot this one. No Blockers, but one "Critical": SPARK-16845 org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB SPARK-18669 Update Apache docs regard watermarking in Structured Streaming SPARK-18894 Event time

Re: [VOTE] Apache Spark 2.1.0 (RC5)

quar.k...@gmail.com>> Sent: Sunday, December 18, 2016 2:33 PM Subject: Re: [VOTE] Apache Spark 2.1.0 (RC5) To: Adam Roberts <arobe...@uk.ibm.com<mailto:arobe...@uk.ibm.com>> Cc: Denny Lee <denny.g@gmail.com<mailto:denny.g@gmail.com>>, Holden Karau <hol...@pigscanfly.

Re: [VOTE] Apache Spark 2.1.0 (RC5)

and > 10% slowdowns for q7, q39a, q43, q52, > q57, q89. Five iterations, average times compared, only changing which > version of Spark we're using > > > > From:Holden Karau <hol...@pigscanfly.ca> > To:Denny Lee <denny.g....@gmail.com>,

Re: [VOTE] Apache Spark 2.1.0 (RC5)

n...@hotmail.com>: For R we have a license field in the DESCRIPTION, and this is standard practice (and requirement) for R packages. https://cran.r-project.org/doc/manuals/R-exts.html#Licensing From: Sean Owen <so...@cloudera.com> Sent: Friday, December 16, 2016 9:57:15

Re: [VOTE] Apache Spark 2.1.0 (RC5)

> > > > > > https://cran.r-project.org/doc/manuals/R-exts.html#Licensing > > > > > > > > -- > > > *From:* Sean Owen <so...@cloudera.com> > > > *Sent:* Friday, December 16, 2016 9:57:15 AM > > > *To

Re: [VOTE] Apache Spark 2.1.0 (RC5)

s is standard > practice (and requirement) for R packages. > > https://cran.r-project.org/doc/manuals/R-exts.html#Licensing > > -- > *From:* Sean Owen <so...@cloudera.com> > *Sent:* Friday, December 16, 2016 9:57:15 AM > *To:* Reynold Xin; dev@spark.

Re: [VOTE] Apache Spark 2.1.0 (RC5)

t; >>>> -------------- >>>> *From:* Sean Owen <so...@cloudera.com> >>>> *Sent:* Friday, December 16, 2016 9:57:15 AM >>>> *To:* Reynold Xin; dev@spark.apache.org >>>> *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5)

Re: [VOTE] Apache Spark 2.1.0 (RC5)

; >>> -- >>> *From:* Sean Owen <so...@cloudera.com> >>> *Sent:* Friday, December 16, 2016 9:57:15 AM >>> *To:* Reynold Xin; dev@spark.apache.org >>> *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) >>> >>

Re: [VOTE] Apache Spark 2.1.0 (RC5)

practice (and requirement) for R packages. >> >> https://cran.r-project.org/doc/manuals/R-exts.html#Licensing >> >> -- >> *From:* Sean Owen <so...@cloudera.com> >> *Sent:* Friday, December 16, 2016 9:57:15 AM >> *To:* Reynold Xin;

Re: [VOTE] Apache Spark 2.1.0 (RC5)

xts.html#Licensing > > -- > *From:* Sean Owen <so...@cloudera.com> > *Sent:* Friday, December 16, 2016 9:57:15 AM > *To:* Reynold Xin; dev@spark.apache.org > *Subject:* Re: [VOTE] Apache Spark 2.1.0 (RC5) > > (If you have a template for these emails, maybe update it to use

Re: [VOTE] Apache Spark 2.1.0 (RC5)

RC5 is also tested on CentOS 6.8, OpenJDK 1.8.0_111, R 3.3.2 with profiles `-Pyarn -Phadoop-2.7 -Pkinesis-asl -Phive -Phive-thriftserver -Psparkr`. BTW, there still exist five on-going issues in JIRA (with target version 2.1.0). 1. SPARK-16845

Re: [VOTE] Apache Spark 2.1.0 (RC5)

(If you have a template for these emails, maybe update it to use https links. They work for apache.org domains. After all we are asking people to verify the integrity of release artifacts, so it might as well be secure.) (Also the new archives use .tar.gz instead of .tgz like the others. No big

Re: [VOTE] Apache Spark 2.1.0 (RC5)

Thanks for the specific mention of the new PySpark packaging Shivaram, For *nix (Linux, Unix, OS X, etc.) Python users interested in helping test the new artifacts you can do as follows: Setup PySpark with pip by: 1. Download the artifact from

Re: [VOTE] Apache Spark 2.1.0 (RC5)

I'm going to start this with a +1! On Thu, Dec 15, 2016 at 9:42 PM, Shivaram Venkataraman < shiva...@eecs.berkeley.edu> wrote: > In addition to usual binary artifacts, this is the first release where > we have installable packages for Python [1] and R [2] that are part of > the release. I'm

Re: [VOTE] Apache Spark 2.1.0 (RC5)

In addition to usual binary artifacts, this is the first release where we have installable packages for Python [1] and R [2] that are part of the release. I'm including instructions to test the R package below. Holden / other Python developers can chime in if there are special instructions to

[VOTE] Apache Spark 2.1.0 (RC5)

Please vote on releasing the following candidate as Apache Spark version 2.1.0. The vote is open until Sun, December 18, 2016 at 21:30 PT and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Spark 2.1.0 [ ] -1 Do not release this package because ...