git commit: Proper indent for the previous commit.

2014-09-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master feaa3706f -> b4dded40f Proper indent for the previous commit. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/b4dded40 Tree: http://git-wip-us.apache.org/repos/asf/spar

git commit: SPARK-3470 [CORE] [STREAMING] Add Closeable / close() to Java context objects

2014-09-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master e11eeb71f -> feaa3706f SPARK-3470 [CORE] [STREAMING] Add Closeable / close() to Java context objects ... that expose a stop() lifecycle method. This doesn't add `AutoCloseable`, which is Java 7+ only. But it should be possible to use try-

git commit: [SQL][Docs] Update SQL programming guide to show the correct default value of containsNull in an ArrayType

2014-09-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master 2584ea5b2 -> e11eeb71f [SQL][Docs] Update SQL programming guide to show the correct default value of containsNull in an ArrayType After #1889, the default value of `containsNull` in an `ArrayType` is `true`. Author: Yin Huai Closes #237

git commit: [SPARK-3469] Make sure all TaskCompletionListener are called even with failures

2014-09-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6d887db78 -> 2584ea5b2 [SPARK-3469] Make sure all TaskCompletionListener are called even with failures This is necessary because we rely on this callback interface to clean resources up. The old behavior would lead to resource leaks. Note

git commit: [SPARK-3515][SQL] Moves test suite setup code to beforeAll rather than in constructor

2014-09-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.1 9c06c7230 -> 44e534eb2 [SPARK-3515][SQL] Moves test suite setup code to beforeAll rather than in constructor Please refer to the JIRA ticket for details. **NOTE** We should check all test suites that do similar initialization-like si

git commit: [SPARK-3515][SQL] Moves test suite setup code to beforeAll rather than in constructor

2014-09-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 885d1621b -> 6d887db78 [SPARK-3515][SQL] Moves test suite setup code to beforeAll rather than in constructor Please refer to the JIRA ticket for details. **NOTE** We should check all test suites that do similar initialization-like side e

git commit: [SPARK-3500] [SQL] use JavaSchemaRDD as SchemaRDD._jschema_rdd

2014-09-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.1 6cbf83c05 -> 9c06c7230 [SPARK-3500] [SQL] use JavaSchemaRDD as SchemaRDD._jschema_rdd Currently, SchemaRDD._jschema_rdd is SchemaRDD, the Scala API (coalesce(), repartition()) can not been called in Python easily, there is no way to sp

git commit: [SPARK-3500] [SQL] use JavaSchemaRDD as SchemaRDD._jschema_rdd

2014-09-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 71af030b4 -> 885d1621b [SPARK-3500] [SQL] use JavaSchemaRDD as SchemaRDD._jschema_rdd Currently, SchemaRDD._jschema_rdd is SchemaRDD, the Scala API (coalesce(), repartition()) can not been called in Python easily, there is no way to specif

git commit: [SPARK-3094] [PySpark] compatitable with PyPy

2014-09-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 25311c2c5 -> 71af030b4 [SPARK-3094] [PySpark] compatitable with PyPy After this patch, we can run PySpark in PyPy (testing with PyPy 2.3.1 in Mac 10.9), for example: ``` PYSPARK_PYTHON=pypy ./bin/spark-submit wordcount.py ``` The perform

git commit: [SPARK-3456] YarnAllocator on alpha can lose container requests to RM

2014-09-12 Thread tgraves
Repository: spark Updated Branches: refs/heads/master af2583826 -> 25311c2c5 [SPARK-3456] YarnAllocator on alpha can lose container requests to RM Author: Thomas Graves Closes #2373 from tgravescs/SPARK-3456 and squashes the following commits: 77e9532 [Thomas Graves] [SPARK-3456] YarnAlloca

git commit: [SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set.

2014-09-12 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 1d767967e -> af2583826 [SPARK-3217] Add Guava to classpath when SPARK_PREPEND_CLASSES is set. When that option is used, the compiled classes from the build directory are prepended to the classpath. Now that we avoid packaging Guava, that me

git commit: SPARK-3014. Log a more informative messages in a couple failure scenario...

2014-09-12 Thread tgraves
Repository: spark Updated Branches: refs/heads/master 15a564598 -> 1d767967e SPARK-3014. Log a more informative messages in a couple failure scenario... ...s Author: Sandy Ryza Closes #1934 from sryza/sandy-spark-3014 and squashes the following commits: ae19cc1 [Sandy Ryza] SPARK-3014. Log

git commit: [SPARK-3427] [GraphX] Avoid active vertex tracking in static PageRank

2014-09-12 Thread rxin
Repository: spark Updated Branches: refs/heads/master eae81b0bf -> 15a564598 [SPARK-3427] [GraphX] Avoid active vertex tracking in static PageRank GraphX's current implementation of static (fixed iteration count) PageRank uses the Pregel API. This unnecessarily tracks active vertices, even th

git commit: MAINTENANCE: Automated closing of pull requests.

2014-09-12 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 8194fc662 -> eae81b0bf MAINTENANCE: Automated closing of pull requests. This commit exists to close the following pull requests on Github: Closes #930 (close requested by 'andrewor14') Closes #867 (close requested by 'marmbrus') Closes #18

svn commit: r1624639 - in /spark: downloads.md js/downloads.js site/downloads.html site/js/downloads.js

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 20:36:11 2014 New Revision: 1624639 URL: http://svn.apache.org/r1624639 Log: Adding pretty titles for download links. Modified: spark/downloads.md spark/js/downloads.js spark/site/downloads.html spark/site/js/downloads.js Modified: spark/downloa

git commit: [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test

2014-09-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.0 ae6f5545a -> 4f991c9ff [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test Logically, we should remove the Hive Table/Database first and then reset the Hive configuration, repoint to the new data warehouse director

git commit: [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test

2014-09-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.1 f17b7957a -> 6cbf83c05 [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test Logically, we should remove the Hive Table/Database first and then reset the Hive configuration, repoint to the new data warehouse director

git commit: [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test

2014-09-12 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 533377621 -> 8194fc662 [SPARK-3481] [SQL] Eliminate the error log in local Hive comparison test Logically, we should remove the Hive Table/Database first and then reset the Hive configuration, repoint to the new data warehouse directory et

git commit: Revert "[Spark-3490] Disable SparkUI for tests"

2014-09-12 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.1 e69deb818 -> f17b7957a Revert "[Spark-3490] Disable SparkUI for tests" This reverts commit 2ffc7980c6818eec05e32141c52e335bc71daed9. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos

git commit: [PySpark] Add blank line so that Python RDD.top() docstring renders correctly

2014-09-12 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master f116f76bf -> 533377621 [PySpark] Add blank line so that Python RDD.top() docstring renders correctly Author: RJ Nowling Closes #2370 from rnowling/python_rdd_docstrings and squashes the following commits: 5230574 [RJ Nowling] Add blank

svn commit: r1624579 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 16:04:01 2014 New Revision: 1624579 URL: http://svn.apache.org/r1624579 Log: Adding missing release credit Modified: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Modified: spark/releases/_posts/2014

svn commit: r1624578 - in /spark: releases/_posts/2014-09-11-spark-release-1-1-0.md site/releases/spark-release-1-1-0.html

2014-09-12 Thread pwendell
Author: pwendell Date: Fri Sep 12 16:03:14 2014 New Revision: 1624578 URL: http://svn.apache.org/r1624578 Log: Adding missing release credit Modified: spark/releases/_posts/2014-09-11-spark-release-1-1-0.md spark/site/releases/spark-release-1-1-0.html Modified: spark/releases/_posts/2014

git commit: [SPARK-2558][DOCS] Add --queue example to YARN doc

2014-09-12 Thread tgraves
Repository: spark Updated Branches: refs/heads/master b8634df1f -> f116f76bf [SPARK-2558][DOCS] Add --queue example to YARN doc Put original YARN queue spark-submit arg description in running-on-yarn html table and example command line Author: Mark G. Whitney Closes #2218 from kramimus/2258

git commit: [SPARK-3160] [SPARK-3494] [mllib] DecisionTree: eliminate pre-allocated nodes, parentImpurities arrays. Memory calc bug fix.

2014-09-12 Thread meng
Repository: spark Updated Branches: refs/heads/master 42904b8d0 -> b8634df1f [SPARK-3160] [SPARK-3494] [mllib] DecisionTree: eliminate pre-allocated nodes, parentImpurities arrays. Memory calc bug fix. This PR includes some code simplifications and re-organization which will be helpful for