spark git commit: [MINOR][MAINTENANCE] Fix typo for the pull request template.

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master 86c852cf2 -> 831429170 [MINOR][MAINTENANCE] Fix typo for the pull request template. ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was the this patch tested? (Please explain how t

spark git commit: [SPARK-12759][Core][Spark should fail fast if --executor-memory is too small for spark to start]

2016-02-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master 831429170 -> bcfd55fa9 [SPARK-12759][Core][Spark should fail fast if --executor-memory is too small for spark to start] Added an exception to be thrown in UnifiedMemoryManager.scala if the configuration given for executor memory is too lo

spark git commit: [SPARK-13390][SQL][BRANCH-1.6] Fix the issue that Iterator.map().toSeq is not Serializable

2016-02-24 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 573a2c97e -> 06f4fce29 [SPARK-13390][SQL][BRANCH-1.6] Fix the issue that Iterator.map().toSeq is not Serializable ## What changes were proposed in this pull request? `scala.collection.Iterator`'s methods (e.g., map, filter) will retur

spark git commit: [SPARK-13472] [SPARKR] Fix unstable Kmeans test in R

2016-02-24 Thread meng
Repository: spark Updated Branches: refs/heads/master bcfd55fa9 -> 893018183 [SPARK-13472] [SPARKR] Fix unstable Kmeans test in R JIRA: https://issues.apache.org/jira/browse/SPARK-13472 ## What changes were proposed in this pull request? One Kmeans test in R is unstable and sometimes fails.

spark git commit: [SPARK-13383][SQL] Keep broadcast hint after column pruning

2016-02-24 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 893018183 -> f37398699 [SPARK-13383][SQL] Keep broadcast hint after column pruning JIRA: https://issues.apache.org/jira/browse/SPARK-13383 ## What changes were proposed in this pull request? When we do column pruning in Optimizer, we put

[1/2] spark git commit: Revert "[SPARK-13383][SQL] Keep broadcast hint after column pruning"

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master f37398699 -> d563c8fa0 Revert "[SPARK-13383][SQL] Keep broadcast hint after column pruning" This reverts commit f3739869973ba4285196a61775d891292b8e282b. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us

[2/2] spark git commit: Revert "[SPARK-13376] [SQL] improve column pruning"

2016-02-24 Thread rxin
Revert "[SPARK-13376] [SQL] improve column pruning" This reverts commit e9533b419e3a87589313350310890ce0caf73dbb. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d563c8fa Tree: http://git-wip-us.apache.org/repos/asf/spark/tre

spark git commit: Revert "Revert "[SPARK-13383][SQL] Keep broadcast hint after column pruning""

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master d563c8fa0 -> 65805ab6e Revert "Revert "[SPARK-13383][SQL] Keep broadcast hint after column pruning"" This reverts commit 382b27babf7771b724f7abff78195a858631d138. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://g

spark git commit: [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization

2016-02-24 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 ed3c1d700 -> a9b1b8025 [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization Changes the personalized pagerank initialization to be non-uniform. Author: Yves Raimond Closes #9386 from moustaki/personalized-

spark git commit: [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization

2016-02-24 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.5 4695eb832 -> d2c1c67cf [SPARK-11432][GRAPHX] Personalized PageRank shouldn't use uniform initialization Changes the personalized pagerank initialization to be non-uniform. Author: Yves Raimond Closes #9386 from moustaki/personalized-

spark git commit: Revert "[SPARK-13321][SQL] Support nested UNION in parser"

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master 65805ab6e -> f92f53fae Revert "[SPARK-13321][SQL] Support nested UNION in parser" This reverts commit 55d6fdf22d1d6379180ac09f364c38982897d9ff. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.or

spark git commit: [SPARK-13467] [PYSPARK] abstract python function to simplify pyspark code

2016-02-24 Thread davies
Repository: spark Updated Branches: refs/heads/master f92f53fae -> a60f91284 [SPARK-13467] [PYSPARK] abstract python function to simplify pyspark code ## What changes were proposed in this pull request? When we pass a Python function to JVM side, we also need to send its context, e.g. `envVa

spark git commit: [HOT][TEST] Disable a Test that Requires Nested Union Support.

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master a60f91284 -> 5289837a7 [HOT][TEST] Disable a Test that Requires Nested Union Support. ## What changes were proposed in this pull request? Since "[SPARK-13321][SQL] Support nested UNION in parser" is reverted, we need to disable the test ca

spark git commit: [SPARK-13475][TESTS][SQL] HiveCompatibilitySuite should still run in PR builder even if a PR only changes sql/core

2016-02-24 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 5289837a7 -> bc353805b [SPARK-13475][TESTS][SQL] HiveCompatibilitySuite should still run in PR builder even if a PR only changes sql/core ## What changes were proposed in this pull request? `HiveCompatibilitySuite` should still run in PR

spark git commit: [SPARK-13475][TESTS][SQL] HiveCompatibilitySuite should still run in PR builder even if a PR only changes sql/core

2016-02-24 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.6 06f4fce29 -> fe71cabd4 [SPARK-13475][TESTS][SQL] HiveCompatibilitySuite should still run in PR builder even if a PR only changes sql/core ## What changes were proposed in this pull request? `HiveCompatibilitySuite` should still run in

svn commit: r1732240 [2/2] - in /spark: _layouts/ site/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2016-02-24 Thread yhuai
Modified: spark/site/news/two-weeks-to-spark-summit-2014.html URL: http://svn.apache.org/viewvc/spark/site/news/two-weeks-to-spark-summit-2014.html?rev=1732240&r1=1732239&r2=1732240&view=diff == --- spark/site/news/two-wee

svn commit: r1732240 [1/2] - in /spark: _layouts/ site/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2016-02-24 Thread yhuai
Author: yhuai Date: Wed Feb 24 23:07:27 2016 New Revision: 1732240 URL: http://svn.apache.org/viewvc?rev=1732240&view=rev Log: Add links to ASF on the nav bar Modified: spark/_layouts/global.html spark/site/community.html spark/site/documentation.html spark/site/downloads.html

spark git commit: [SPARK-13383][SQL] Fix test

2016-02-24 Thread yhuai
Repository: spark Updated Branches: refs/heads/master bc353805b -> cbb0b65ad [SPARK-13383][SQL] Fix test ## What changes were proposed in this pull request? Reverting SPARK-13376 (https://github.com/apache/spark/commit/d563c8fa01cfaebb5899ff7970115d0f2e64e8d5) affects the test added by SPAR

spark git commit: [SPARK-13250] [SQL] Update PhysicallRDD to convert to UnsafeRow if using the vectorized scanner.

2016-02-24 Thread davies
Repository: spark Updated Branches: refs/heads/master cbb0b65ad -> 5a7af9e7a [SPARK-13250] [SQL] Update PhysicallRDD to convert to UnsafeRow if using the vectorized scanner. Some parts of the engine rely on UnsafeRow which the vectorized parquet scanner does not want to produce. This add a c

spark git commit: [SPARK-13092][SQL] Add ExpressionSet for constraint tracking

2016-02-24 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 5a7af9e7a -> 2b042577f [SPARK-13092][SQL] Add ExpressionSet for constraint tracking This PR adds a new abstraction called an `ExpressionSet` which attempts to canonicalize expressions to remove cosmetic differences. Deterministic express

spark git commit: [SPARK-13479][SQL][PYTHON] Added Python API for approxQuantile

2016-02-24 Thread meng
Repository: spark Updated Branches: refs/heads/master 2b042577f -> 13ce10e95 [SPARK-13479][SQL][PYTHON] Added Python API for approxQuantile ## What changes were proposed in this pull request? * Scala DataFrameStatFunctions: Added version of approxQuantile taking a List instead of an Array, f

spark git commit: [SPARK-7106][MLLIB][PYSPARK] Support model save/load in Python's FPGrowth

2016-02-24 Thread meng
Repository: spark Updated Branches: refs/heads/master 13ce10e95 -> 4d2864b2d [SPARK-7106][MLLIB][PYSPARK] Support model save/load in Python's FPGrowth ## What changes were proposed in this pull request? Python API supports mode save/load in FPGrowth JIRA: [https://issues.apache.org/jira/brow

spark git commit: [SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf.

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.6 fe71cabd4 -> 897599601 [SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf. `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024 as integer and the other one is

spark git commit: [SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf.

2016-02-24 Thread rxin
Repository: spark Updated Branches: refs/heads/master 4d2864b2d -> 264533b55 [SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf. `spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024 as integer and the other one is '2m'