[RESULT] [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-03-03 Thread Patrick Wendell
This vote is cancelled in favor of RC2. On Thu, Feb 26, 2015 at 9:50 AM, Sandor Van Wassenhove sand...@palantir.com wrote: FWIW, I tested the first rc and saw no regressions. I ran our benchmarks built against spark 1.3 and saw results consistent with spark 1.2/1.2.1. On 2/25/15, 5:51 PM,

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-26 Thread Sandor Van Wassenhove
FWIW, I tested the first rc and saw no regressions. I ran our benchmarks built against spark 1.3 and saw results consistent with spark 1.2/1.2.1. On 2/25/15, 5:51 PM, Patrick Wendell pwend...@gmail.com wrote: Hey All, Just a quick updated on this thread. Issues have continued to trickle in. Not

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-25 Thread Patrick Wendell
Hey All, Just a quick updated on this thread. Issues have continued to trickle in. Not all of them are blocker level but enough to warrant another RC: I've been keeping the JIRA dashboard up and running with the latest status (sorry, long link):

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Robin East
Running ec2 launch scripts gives me the following error: ssl.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed Full stack trace at https://gist.github.com/insidedctm/4d41600bc22560540a26 I’m running OSX Mavericks 10.9.5 I’ll

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Corey Nolet
This vote was supposed to close on Saturday but it looks like no PMCs voted (other than the implicit vote from Patrick). Was there a discussion offline to cut an RC2? Was the vote extended? On Mon, Feb 23, 2015 at 6:59 AM, Robin East robin.e...@xense.co.uk wrote: Running ec2 launch scripts

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Corey Nolet
Thanks Sean. I glossed over the comment about SPARK-5669. On Mon, Feb 23, 2015 at 9:05 AM, Sean Owen so...@cloudera.com wrote: Yes my understanding from Patrick's comment is that this RC will not be released, but, to keep testing. There's an implicit -1 out of the gates there, I believe, and

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Patrick Wendell
So actually, the list of blockers on JIRA is a bit outdated. These days I won't cut RC1 unless there are no known issues that I'm aware of that would actually block the release (that's what the snapshot ones are for). I'm going to clean those up and push others to do so also. The main issues I'm

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Patrick Wendell
It's only been reported on this thread by Tom, so far. On Mon, Feb 23, 2015 at 10:29 AM, Marcelo Vanzin van...@cloudera.com wrote: Hey Patrick, Do you have a link to the bug related to Python and Yarn? I looked at the blockers in Jira but couldn't find it. On Mon, Feb 23, 2015 at 10:18 AM,

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Marcelo Vanzin
Hey Patrick, Do you have a link to the bug related to Python and Yarn? I looked at the blockers in Jira but couldn't find it. On Mon, Feb 23, 2015 at 10:18 AM, Patrick Wendell pwend...@gmail.com wrote: So actually, the list of blockers on JIRA is a bit outdated. These days I won't cut RC1

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Marcelo Vanzin
Hi Tom, are you using an sbt-built assembly by any chance? If so, take a look at SPARK-5808. I haven't had any problems with the maven-built assembly. Setting SPARK_HOME on the executors is a workaround if you want to use the sbt assembly. On Fri, Feb 20, 2015 at 2:56 PM, Tom Graves

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Michael Armbrust
On Sun, Feb 22, 2015 at 11:20 PM, Mark Hamstra m...@clearstorydata.com wrote: So what are we expecting of Hive 0.12.0 builds with this RC? I know not every combination of Hadoop and Hive versions, etc., can be supported, but even an example build from the Building Spark page isn't looking too

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Mark Hamstra
Nothing that I can point to, so this may only be a problem in test scope. I am looking at a problem where some UDFs that run with 0.12 fail with 0.13; but that problem is already present in Spark 1.2.x, so it's not a blocking regression for 1.3. (Very likely a HiveFunctionWrapper serde problem,

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Soumitra Kumar
+1 (non-binding) For: https://issues.apache.org/jira/browse/SPARK-3660 . Docs OK . Example code is good -Soumitra. On Mon, Feb 23, 2015 at 10:33 AM, Marcelo Vanzin van...@cloudera.com wrote: Hi Tom, are you using an sbt-built assembly by any chance? If so, take a look at SPARK-5808. I

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Tathagata Das
Hey all, I found a major issue where JobProgressListener (a listener used to keep track of jobs for the web UI) never forgets stages in one of its data structures. This is a blocker for long running applications. https://issues.apache.org/jira/browse/SPARK-5967 I am testing a fix for this right

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-23 Thread Cheng Lian
My bad, had once fixed all Hive 12 test failures in PR #4107, but didn't got time to get it merged. Considering the release is close, I can cherry-pick those Hive 12 fixes from #4107 and open a more surgical PR soon. Cheng On 2/24/15 4:18 AM, Michael Armbrust wrote: On Sun, Feb 22, 2015 at

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-22 Thread Mark Hamstra
So what are we expecting of Hive 0.12.0 builds with this RC? I know not every combination of Hadoop and Hive versions, etc., can be supported, but even an example build from the Building Spark page isn't looking too good to me. Working from f97b0d4, the example build command works: mvn -Pyarn

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-20 Thread Tom Graves
Trying to run pyspark on yarn in client mode with basic wordcount example I see the following error when doing the collect: Error from python worker:  /usr/bin/python: No module named sqlPYTHONPATH was: 

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-19 Thread Michael Armbrust
P.S: For some reason replacing import sqlContext.createSchemaRDD with import sqlContext.implicits._ doesn't do the implicit conversations. registerTempTable gives syntax error. I will dig deeper tomorrow. Has anyone seen this ? We will write up a whole migration guide before the final

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-19 Thread Krishna Sankar
Excellent. Explicit toDF() works. a) employees.toDF().registerTempTable(Employees) - works b) Also affects saveAsParquetFile - orders.toDF().saveAsParquetFile Adding to my earlier tests: 4.0 SQL from Scala and Python 4.1 result = sqlContext.sql(SELECT * from Employees WHERE State = 'WA') OK 4.2

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-19 Thread Timothy Chen
+1 (non-binding) Tested Mesos coarse/fine-grained mode with 4 nodes Mesos cluster with simple shuffle/map task. Will be testing with more complete suite (ie: spark-perf) once the infrastructure is setup to do so. Tim On Thu, Feb 19, 2015 at 12:50 PM, Krishna Sankar ksanka...@gmail.com wrote:

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-19 Thread Corey Nolet
+1 (non-binding) - Verified signatures using [1] - Built on MacOSX Yosemite - Built on Fedora 21 Each build was run with and Hadoop-2.4 version with yarn, hive, and hive-thriftserver profiles I am having trouble getting all the tests passing on a single run on both machines but we have this

[VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-18 Thread Patrick Wendell
Please vote on releasing the following candidate as Apache Spark version 1.3.0! The tag to be voted on is v1.3.0-rc1 (commit f97b0d4a): https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=f97b0d4a6b26504916816d7aefcf3132cd1da6c2 The release files, including signatures, digests, etc.

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-18 Thread Patrick Wendell
UISeleniumSuite: *** RUN ABORTED *** java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal ... This is a newer test suite. There is something flaky about it, we should definitely fix it, IMO it's not a blocker though. Patrick this link gives a 404:

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-18 Thread Sean Owen
On OS X and Ubuntu I see the following test failure in the source release for 1.3.0-RC1: UISeleniumSuite: *** RUN ABORTED *** java.lang.NoClassDefFoundError: org/w3c/dom/ElementTraversal ... Patrick this link gives a 404: https://people.apache.org/keys/committer/pwendell.asc Finally, I

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-18 Thread Sean Owen
On Wed, Feb 18, 2015 at 6:13 PM, Patrick Wendell pwend...@gmail.com wrote: Patrick this link gives a 404: https://people.apache.org/keys/committer/pwendell.asc Works for me. Maybe it's some ephemeral issue? Yes works now; I swear it didn't before! that's all set now. The signing key is in

Re: [VOTE] Release Apache Spark 1.3.0 (RC1)

2015-02-18 Thread Krishna Sankar
+1 (non-binding, of course) 1. Compiled OSX 10.10 (Yosemite) OK Total time: 14:50 min mvn clean package -Pyarn -Dyarn.version=2.6.0 -Phadoop-2.4 -Dhadoop.version=2.6.0 -Phive -DskipTests -Dscala-2.11 2. Tested pyspark, mlib - running as well as compare results with 1.1.x 1.2.x 2.1.