Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-16 Thread Matei Zaharia
-Original Message- From: Liu, Raymond [mailto:raymond@intel.com] Sent: Monday, December 16, 2013 10:48 AM To: dev@spark.incubator.apache.org Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-15 Thread Patrick Wendell
You can checkout the docs mentioned in the vote thread. There is also a pre-build binary for hadoop2 that is compiled for YARN 2.2 - Patrick On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu azury...@gmail.com wrote: yarn 2.2, not yarn 0.22, I am so sorry. On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu

RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-15 Thread Liu, Raymond
: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi here, Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions. Hadoop-2.x using protobuf-2.5.0 but if some guys want to run Spark on mesos, then mesos using protobuf-2.4.1 currently

RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-15 Thread Liu, Raymond
To: dev@spark.incubator.apache.org Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue Best Regards, Raymond Liu -Original Message- From: Azuryy Yu [mailto:azury

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-15 Thread Matei Zaharia
: Liu, Raymond [mailto:raymond@intel.com] Sent: Monday, December 16, 2013 10:48 AM To: dev@spark.incubator.apache.org Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-15 Thread Azuryy Yu
, Raymond [mailto:raymond@intel.com] Sent: Monday, December 16, 2013 10:48 AM To: dev@spark.incubator.apache.org Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-14 Thread Henry Saputra
Hi Patrick, as sebb has mentioned let's move the binaries from the voting directory in your people.apache.org directory. ASF release voting is for source code and not binaries, and technically we provide binaries for convenience. And add link to the KEYS location in the dist[1] to let verify

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-14 Thread Henry Saputra
Actually we should be fine putting the binaries there as long as the VOTE is for the source. Let's verify with sebb in the general@ list about his concern. - Henry On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra henry.sapu...@gmail.com wrote: Hi Patrick, as sebb has mentioned let's move the

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-13 Thread Patrick Wendell
The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 votes. +1 (5 Total) Matei Zaharia* Nick Pentreath* Patrick Wendell* Prashant Sharma* Tom Graves* 0 (0 Total) -1 (0 Total) * = Binding Vote As per the incubator release guide [1] I'll be sending this to the general

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Prashant Sharma
+1, built and tested on linux. On Thu, Dec 12, 2013 at 5:57 AM, Patrick Wendell pwend...@gmail.com wrote: I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they re-downloaded it worked. So maybe just

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Tom Graves
I re-downloaded the source tarball and it works now. Tom On Wednesday, December 11, 2013 6:27 PM, Patrick Wendell pwend...@gmail.com wrote: I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Tom Graves
+1.  Built spark on yarn for both hadoop 0.23 and hadoop 2.2.0 on redhat linux using maven.  Ran some tests on both a secure Hadoop 0.23 cluster and a secure Hadoop 2.2.0 cluster.  Verified signatures and md5. Tom On Tuesday, December 10, 2013 6:49 PM, Patrick Wendell pwend...@gmail.com

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-12 Thread Evan Chan
I'd be personally fine with a standard workflow of assemble-deps + packaging just the Spark files as separate packages, if it speeds up everyone's development time. On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra m...@clearstorydata.comwrote: I don't know how to make sense of the numbers, but

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Nick Pentreath
- Successfully built via sbt/sbt assembly/assembly on Mac OS X, as well as on a dev Ubuntu EC2 box - Successfully tested via sbt/sbt test locally - Successfully built and tested using mvn package locally - I've tested my own Spark jobs (built against 0.8.0-incubating) on this RC

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Patrick Wendell
I'll +1 myself also. For anyone who has the slow build problem: does this issue happen when building v0.8.0-incubating also? Trying to figure out whether it's related to something we added in 0.8.1 or if it's a long standing issue. - Patrick On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Taka Shinagawa
On my Core 2 Duo (slower) Mac, I'm having the same problem with building v0.8.0-incubating as well. I'm suspecting the slow build problem is caused by sbt-assembly as reported here. https://github.com/sbt/sbt-assembly/issues/68 I tried to upgrade sbt-assembly to version 0.10.1 (and sbt to

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Patrick Wendell
Hey Tom, I re-verified the signatures and got someone else to do it. It seemed fine. Here is what I did. gpg --recv-key 9E4FE3AF wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc wget

Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)

2013-12-11 Thread Patrick Wendell
I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they re-downloaded it worked. So maybe just re-download and try again? On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell pwend...@gmail.com wrote: Hey