Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-11 Thread Matei Zaharia
Unless you can diagnose the problem quickly, Gary, I think we need to go ahead with this release as is. This release didn't touch the Mesos support as far as I know, so the problem might be a nondeterministic issue with your application. But on the other hand the release does fix some critical

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-11 Thread Gary Malouf
Hi Matei, We have not had time to re-deploy the rc today, but one thing that jumps out is the shrinking of the default akka frame size from 10MB to around 128KB by default. That is my first suspicion for our issue - could imagine that biting others as well. I'll try to re-test that today -

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-11 Thread Patrick Wendell
Hey Gary, Why do you think the akka frame size changed? It didn't change - we added some fixes for cases where users were setting non-default values. On Fri, Jul 11, 2014 at 9:31 AM, Gary Malouf malouf.g...@gmail.com wrote: Hi Matei, We have not had time to re-deploy the rc today, but one

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-11 Thread Patrick Wendell
Okay just FYI - I'm closing this vote since many people are waiting on the release and I was hoping to package it today. If we find a reproducible Mesos issue here, we can definitely spin the fix into a subsequent release. On Fri, Jul 11, 2014 at 9:37 AM, Patrick Wendell pwend...@gmail.com

[RESULT] [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-11 Thread Patrick Wendell
This vote has passed with 9 +1 votes (5 binding) and 1 -1 vote (0 binding). +1: Patrick Wendell* Mark Hamstra* DB Tsai Krishna Sankar Soren Macbeth Andrew Or Matei Zaharia* Xiangrui Meng* Tom Graves* 0: -1: Gary Malouf

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-10 Thread Gary Malouf
-1 I honestly do not know the voting rules for the Spark community, so please excuse me if I am out of line or if Mesos compatibility is not a concern at this point. We just tried to run this version built against 2.3.0-cdh5.0.2 on mesos 0.18.2. All of our jobs with data above a few gigabytes

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-10 Thread Gary Malouf
Just realized the deadline was Monday, my apologies. The issue nevertheless stands. On Thu, Jul 10, 2014 at 9:28 PM, Gary Malouf malouf.g...@gmail.com wrote: -1 I honestly do not know the voting rules for the Spark community, so please excuse me if I am out of line or if Mesos compatibility

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-07 Thread Xiangrui Meng
+1 Ran mllib examples. On Sun, Jul 6, 2014 at 1:21 PM, Matei Zaharia matei.zaha...@gmail.com wrote: +1 Tested on Mac OS X. Matei On Jul 6, 2014, at 1:54 AM, Andrew Or and...@databricks.com wrote: +1, verified that the UI bug is in fact fixed in

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-07 Thread Tom Graves
+1. Ran some Spark on yarn jobs on a hadoop 2.4 cluster with authentication on. Tom On Friday, July 4, 2014 2:39 PM, Patrick Wendell pwend...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.0.1! The tag to be voted on is v1.0.1-rc1 (commit

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-06 Thread Matei Zaharia
+1 Tested on Mac OS X. Matei On Jul 6, 2014, at 1:54 AM, Andrew Or and...@databricks.com wrote: +1, verified that the UI bug is in fact fixed in https://github.com/apache/spark/pull/1255. 2014-07-05 20:01 GMT-07:00 Soren Macbeth so...@yieldbot.com: +1 On Sat, Jul 5, 2014 at 7:41

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-05 Thread Mark Hamstra
+1 On Fri, Jul 4, 2014 at 12:40 PM, Patrick Wendell pwend...@gmail.com wrote: I'll start the voting with a +1 - ran tests on the release candidate and ran some basic programs. RC1 passed our performance regression suite, and there are no major changes from that RC. On Fri, Jul 4, 2014 at

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-05 Thread Michael Armbrust
+1 I tested sql/hive functionality. On Sat, Jul 5, 2014 at 9:30 AM, Mark Hamstra m...@clearstorydata.com wrote: +1 On Fri, Jul 4, 2014 at 12:40 PM, Patrick Wendell pwend...@gmail.com wrote: I'll start the voting with a +1 - ran tests on the release candidate and ran some basic

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-05 Thread DB Tsai
+1 On Jul 5, 2014 1:39 PM, Michael Armbrust mich...@databricks.com wrote: +1 I tested sql/hive functionality. On Sat, Jul 5, 2014 at 9:30 AM, Mark Hamstra m...@clearstorydata.com wrote: +1 On Fri, Jul 4, 2014 at 12:40 PM, Patrick Wendell pwend...@gmail.com wrote: I'll start

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-05 Thread Krishna Sankar
+1 - Compiled rc2 w/ CentOS 6.5, Yarn,Hadoop 2.2.0 - successful - Smoke Test (scala,python) (distributed cluster) - successful - We had ran Java/SparkSQL (count, distinct et al) ~250M records RDD over HBase 0.98.3 over last build (rc1) - successful - Stand alone multi-node cluster

Re: [VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-05 Thread Soren Macbeth
+1 On Sat, Jul 5, 2014 at 7:41 PM, Krishna Sankar ksanka...@gmail.com wrote: +1 - Compiled rc2 w/ CentOS 6.5, Yarn,Hadoop 2.2.0 - successful - Smoke Test (scala,python) (distributed cluster) - successful - We had ran Java/SparkSQL (count, distinct et al) ~250M records RDD over

[VOTE] Release Apache Spark 1.0.1 (RC2)

2014-07-04 Thread Patrick Wendell
Please vote on releasing the following candidate as Apache Spark version 1.0.1! The tag to be voted on is v1.0.1-rc1 (commit 7d1043c): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7d1043c99303b87aef8ee19873629c2bfba4cc78 The release files, including signatures, digests, etc.