[VOTE] Release Apache Spark 1.0.0 (RC11)

Please vote on releasing the following candidate as Apache Spark version 1.0.0! This has a few important bug fixes on top of rc10: SPARK-1900 and SPARK-1918: https://github.com/apache/spark/pull/853 SPARK-1870: https://github.com/apache/spark/pull/848 SPARK-1897:

Re: [VOTE] Release Apache Spark 1.0.0 (RC11)

+1 Tested on Mac OS X and Windows. Matei On May 26, 2014, at 7:38 AM, Tathagata Das tathagata.das1...@gmail.com wrote: Please vote on releasing the following candidate as Apache Spark version 1.0.0! This has a few important bug fixes on top of rc10: SPARK-1900 and SPARK-1918:

Spark 1.0: outerJoinVertices seems to return null for vertex attributes when input was partitioned and vertex attribute type is changed

I am seeing something strange with outerJoinVertices(and triangle count that relies on this api): Here is what I am doing: 1) Created a Graph with multiple partitions i.e created a graph with minEdgePartitions(in api GraphLoader.edgeListFile), where minEdgePartitions =1; and use

Re: Spark 1.0: outerJoinVertices seems to return null for vertex attributes when input was partitioned and vertex attribute type is changed

Correction: in step 4) predicate is ed.srcAttr != null ed.dstAttr != null (used -1, when when changed attr type to Int ) -- View this message in context:

Re: Spark 1.0: outerJoinVertices seems to return null for vertex attributes when input was partitioned and vertex attribute type is changed

This is probably due to SPARK-1931https://issues.apache.org/jira/browse/SPARK-1931, which I just fixed in PR #885 https://github.com/apache/spark/pull/885. Is the problem resolved if you use the current Spark master? Ankur http://www.ankurdave.com/

Re: [VOTE] Release Apache Spark 1.0.0 (RC11)

-1 I just fixed SPARK-1931 https://issues.apache.org/jira/browse/SPARK-1931 , which was a critical bug in Graph#partitionBy. Since this is an important part of the GraphX API, I think Spark 1.0.0 should include the fix: https://github.com/apache/spark/pull/885. -- View this message in

Re: [VOTE] Release Apache Spark 1.0.0 (RC11)

Hey Ankur, That does seem like a good fix, but right now we are only blocking the release on major regressions that affect all components. So I don't think this is sufficient to block it from going forward and cutting a new candidate. This is because we are in the very late stage of the release.

Re: [VOTE] Release Apache Spark 1.0.0 (RC11)

I think the question for me would be does this only happen when you call partitionBy, or always? And how common do you expect calls to partitionBy to be? If we can wait for 1.0.1 then I’d wait on this one. Matei On May 26, 2014, at 10:47 PM, Patrick Wendell pwend...@gmail.com wrote: Hey

Re: [VOTE] Release Apache Spark 1.0.0 (RC11)

+1 On Mon, May 26, 2014 at 7:38 AM, Tathagata Das tathagata.das1...@gmail.comwrote: Please vote on releasing the following candidate as Apache Spark version 1.0.0! This has a few important bug fixes on top of rc10: SPARK-1900 and SPARK-1918: https://github.com/apache/spark/pull/853