spark git commit: [MINOR] Fix the a minor bug in PageRank Example.

2015-05-28 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.2 d5763c3b9 - c0a0eaacc [MINOR] Fix the a minor bug in PageRank Example. Fix the bug that entering only 1 arg will cause array out of bounds exception in PageRank example. Author: Li Yao hnkfli...@gmail.com Closes #6455 from

spark git commit: [DOCS] Fixing broken IDE setup link in the Building Spark documentation.

2015-05-28 Thread srowen
Repository: spark Updated Branches: refs/heads/master c771589c9 - 3e312a5ed [DOCS] Fixing broken IDE setup link in the Building Spark documentation. The location of the IDE setup information has changed, so this just updates the link on the Building Spark page. Author: Mike Dusenberry

spark git commit: [MINOR] Fix the a minor bug in PageRank Example.

2015-05-28 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.3 33e1539b3 - 7a3feb552 [MINOR] Fix the a minor bug in PageRank Example. Fix the bug that entering only 1 arg will cause array out of bounds exception in PageRank example. Author: Li Yao hnkfli...@gmail.com Closes #6455 from

spark git commit: [DOCS] Fixing broken IDE setup link in the Building Spark documentation.

2015-05-28 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.4 448528398 - 0a65224ae [DOCS] Fixing broken IDE setup link in the Building Spark documentation. The location of the IDE setup information has changed, so this just updates the link on the Building Spark page. Author: Mike Dusenberry

spark git commit: [DOCS] Fixing broken IDE setup link in the Building Spark documentation.

2015-05-28 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.3 7a3feb552 - d09a053ec [DOCS] Fixing broken IDE setup link in the Building Spark documentation. The location of the IDE setup information has changed, so this just updates the link on the Building Spark page. Author: Mike Dusenberry

spark git commit: [MINOR] Fix the a minor bug in PageRank Example.

2015-05-28 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.4 7b5dffb80 - 448528398 [MINOR] Fix the a minor bug in PageRank Example. Fix the bug that entering only 1 arg will cause array out of bounds exception in PageRank example. Author: Li Yao hnkfli...@gmail.com Closes #6455 from

spark git commit: [MINOR] Fix the a minor bug in PageRank Example.

2015-05-28 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 530efe3e8 - c771589c9 [MINOR] Fix the a minor bug in PageRank Example. Fix the bug that entering only 1 arg will cause array out of bounds exception in PageRank example. Author: Li Yao hnkfli...@gmail.com Closes #6455 from

spark git commit: [SPARK-7911] [MLLIB] A workaround for VectorUDT serialize (or deserialize) being called multiple times

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 ab62d73dd - 7b5dffb80 [SPARK-7911] [MLLIB] A workaround for VectorUDT serialize (or deserialize) being called multiple times ~~A PythonUDT shouldn't be serialized into external Scala types in PythonRDD. I'm not sure whether this

spark git commit: [SPARK-7911] [MLLIB] A workaround for VectorUDT serialize (or deserialize) being called multiple times

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/master 000df2f0d - 530efe3e8 [SPARK-7911] [MLLIB] A workaround for VectorUDT serialize (or deserialize) being called multiple times ~~A PythonUDT shouldn't be serialized into external Scala types in PythonRDD. I'm not sure whether this should

spark git commit: [SPARK-7853] [SQL] Fix HiveContext in Spark Shell

2015-05-28 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 0077af22c - 572b62caf [SPARK-7853] [SQL] Fix HiveContext in Spark Shell https://issues.apache.org/jira/browse/SPARK-7853 This fixes the problem introduced by my change in https://github.com/apache/spark/pull/6435, which causes that Hive

spark git commit: [SPARK-7853] [SQL] Fix HiveContext in Spark Shell

2015-05-28 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.4 9c2c6b4a6 - 8f4a86eaa [SPARK-7853] [SQL] Fix HiveContext in Spark Shell https://issues.apache.org/jira/browse/SPARK-7853 This fixes the problem introduced by my change in https://github.com/apache/spark/pull/6435, which causes that

spark git commit: [SPARK-7577] [ML] [DOC] add bucketizer doc

2015-05-28 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 572b62caf - 1bd63e82f [SPARK-7577] [ML] [DOC] add bucketizer doc CC jkbradley Author: Xusen Yin yinxu...@gmail.com Closes #6451 from yinxusen/SPARK-7577 and squashes the following commits: e2dc32e [Xusen Yin] rename colums e350e49

spark git commit: [SPARK-7577] [ML] [DOC] add bucketizer doc

2015-05-28 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.4 8f4a86eaa - 7bb445a38 [SPARK-7577] [ML] [DOC] add bucketizer doc CC jkbradley Author: Xusen Yin yinxu...@gmail.com Closes #6451 from yinxusen/SPARK-7577 and squashes the following commits: e2dc32e [Xusen Yin] rename colums e350e49

spark git commit: [SPARK-7927] [MLLIB] Enforce whitespace for more tokens in style checker

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 3479e6a12 - 0c0511506 [SPARK-7927] [MLLIB] Enforce whitespace for more tokens in style checker rxin Author: Xiangrui Meng m...@databricks.com Closes #6481 from mengxr/mllib-scalastyle and squashes the following commits: 3ca4d61

spark git commit: [SPARK-7927] [MLLIB] Enforce whitespace for more tokens in style checker

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 9b692bfdf - 04616b1a2 [SPARK-7927] [MLLIB] Enforce whitespace for more tokens in style checker rxin Author: Xiangrui Meng m...@databricks.com Closes #6481 from mengxr/mllib-scalastyle and squashes the following commits: 3ca4d61

spark git commit: [SPARK-7927] whitespace fixes for SQL core.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 04616b1a2 - ff44c711a [SPARK-7927] whitespace fixes for SQL core. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6477 from

spark git commit: Remove SizeEstimator from o.a.spark package.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 b9bdf12a1 - 9c2c6b4a6 Remove SizeEstimator from o.a.spark package. See comments on https://github.com/apache/spark/pull/3913 Author: Reynold Xin r...@databricks.com Closes #6471 from rxin/sizeestimator and squashes the following

spark git commit: [SPARK-7933] Remove Patrick's username/pw from merge script

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 3b38c06f0 - 3479e6a12 [SPARK-7933] Remove Patrick's username/pw from merge script Looks like this was added by accident when pwendell merged a commit back in September: fe2b1d6a209db9fe96b1c6630677955b94bd48c9 Author: Kay Ousterhout

spark git commit: [SPARK-7933] Remove Patrick's username/pw from merge script

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/master ee6a0e12f - 66c49ed60 [SPARK-7933] Remove Patrick's username/pw from merge script Looks like this was added by accident when pwendell merged a commit back in September: fe2b1d6a209db9fe96b1c6630677955b94bd48c9 Author: Kay Ousterhout

spark git commit: [SPARK-7927] whitespace fixes for streaming.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 1bd63e82f - 3af0b3136 [SPARK-7927] whitespace fixes for streaming. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6475 from

spark git commit: [SPARK-7927] whitespace fixes for streaming.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 7bb445a38 - f4b135337 [SPARK-7927] whitespace fixes for streaming. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6475 from

spark git commit: [SPARK-7927] whitespace fixes for GraphX.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 e3dd2802f - b3a590061 [SPARK-7927] whitespace fixes for GraphX. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6474 from

spark git commit: [SPARK-7927] whitespace fixes for GraphX.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7f7505d8d - b069ad23d [SPARK-7927] whitespace fixes for GraphX. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6474 from

[1/2] spark git commit: [SPARK-7927] whitespace fixes for core.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 22e42e3fe - e3dd2802f http://git-wip-us.apache.org/repos/asf/spark/blob/e3dd2802/core/src/test/scala/org/apache/spark/deploy/worker/WorkerArgumentsTest.scala -- diff

[2/2] spark git commit: [SPARK-7927] whitespace fixes for core.

2015-05-28 Thread rxin
[SPARK-7927] whitespace fixes for core. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6473 from rxin/whitespace-core and squashes the following commits: 058195d [Reynold Xin] Fixed tests. fce11e9

[2/2] spark git commit: [SPARK-7927] whitespace fixes for core.

2015-05-28 Thread rxin
[SPARK-7927] whitespace fixes for core. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6473 from rxin/whitespace-core and squashes the following commits: 058195d [Reynold Xin] Fixed tests. fce11e9

spark git commit: [SPARK-7927] whitespace fixes for Catalyst module.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 142ae52d4 - 22e42e3fe [SPARK-7927] whitespace fixes for Catalyst module. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6476 from

[1/2] spark git commit: [SPARK-7927] whitespace fixes for core.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 8da560d7d - 7f7505d8d http://git-wip-us.apache.org/repos/asf/spark/blob/7f7505d8/core/src/test/scala/org/apache/spark/deploy/worker/WorkerArgumentsTest.scala -- diff --git

spark git commit: [SPARK-7198] [MLLIB] VectorAssembler should output ML attributes

2015-05-28 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.4 0a65224ae - b9bdf12a1 [SPARK-7198] [MLLIB] VectorAssembler should output ML attributes `VectorAssembler` should carry over ML attributes. For unknown attributes, we assume numeric values. This PR handles the following cases: 1.

spark git commit: Remove SizeEstimator from o.a.spark package.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 7859ab659 - 0077af22c Remove SizeEstimator from o.a.spark package. See comments on https://github.com/apache/spark/pull/3913 Author: Reynold Xin r...@databricks.com Closes #6471 from rxin/sizeestimator and squashes the following commits:

spark git commit: [SPARK-7927] whitespace fixes for Hive and ThriftServer.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 3af0b3136 - ee6a0e12f [SPARK-7927] whitespace fixes for Hive and ThriftServer. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6478 from

spark git commit: [SPARK-7927] whitespace fixes for Hive and ThriftServer.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 f4b135337 - 3b38c06f0 [SPARK-7927] whitespace fixes for Hive and ThriftServer. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6478 from

spark git commit: [SPARK-7826] [CORE] Suppress extra calling getCacheLocs.

2015-05-28 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master 66c49ed60 - 9b692bfdf [SPARK-7826] [CORE] Suppress extra calling getCacheLocs. There are too many extra call method `getCacheLocs` for `DAGScheduler`, which includes Akka communication. To improve `DAGScheduler` performance, suppress

spark git commit: [SPARK-7927] whitespace fixes for SQL core.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 0c0511506 - 9b97e95e8 [SPARK-7927] whitespace fixes for SQL core. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6477 from

spark git commit: [SPARK-7927] whitespace fixes for Catalyst module.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 2881d14cb - 8da560d7d [SPARK-7927] whitespace fixes for Catalyst module. So we can enable a whitespace enforcement rule in the style checker to save code review time. Author: Reynold Xin r...@databricks.com Closes #6476 from

spark git commit: [SPARK-7929] Remove Bagel examples whitespace fix for examples.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master ff44c711a - 2881d14cb [SPARK-7929] Remove Bagel examples whitespace fix for examples. Author: Reynold Xin r...@databricks.com Closes #6480 from rxin/whitespace-example and squashes the following commits: 8a4a3d4 [Reynold Xin]

spark git commit: [SPARK-7929] Remove Bagel examples whitespace fix for examples.

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 9b97e95e8 - 142ae52d4 [SPARK-7929] Remove Bagel examples whitespace fix for examples. Author: Reynold Xin r...@databricks.com Closes #6480 from rxin/whitespace-example and squashes the following commits: 8a4a3d4 [Reynold Xin]

spark git commit: [SPARK-7922] [MLLIB] use DataFrames for user/item factors in ALSModel

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 f7cb272b7 - 68559423a [SPARK-7922] [MLLIB] use DataFrames for user/item factors in ALSModel Expose user/item factors in DataFrames. This is to be more consistent with the pipeline API. It also helps maintain consistent APIs across

spark git commit: [SPARK-7922] [MLLIB] use DataFrames for user/item factors in ALSModel

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/master cd3d9a5c0 - db9513789 [SPARK-7922] [MLLIB] use DataFrames for user/item factors in ALSModel Expose user/item factors in DataFrames. This is to be more consistent with the pipeline API. It also helps maintain consistent APIs across

Git Push Summary

2015-05-28 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc3 [created] 2d97d7a0a - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 1.4.0-SNAPSHOT

2015-05-28 Thread pwendell
Preparing development version 1.4.0-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/119c93af Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/119c93af Diff:

spark git commit: [SPARK-7932] Fix misleading scheduler delay visualization

2015-05-28 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/branch-1.4 1d49d8c3f - aee046dfa [SPARK-7932] Fix misleading scheduler delay visualization The existing code rounds down to the nearest percent when computing the proportion of a task's time that was spent on each phase of execution, and then

spark git commit: [SPARK-7932] Fix misleading scheduler delay visualization

2015-05-28 Thread kayousterhout
Repository: spark Updated Branches: refs/heads/master 834e69952 - 04ddcd4db [SPARK-7932] Fix misleading scheduler delay visualization The existing code rounds down to the nearest percent when computing the proportion of a task's time that was spent on each phase of execution, and then

spark git commit: [SPARK-7930] [CORE] [STREAMING] Fixed shutdown hook priorities

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/master 04ddcd4db - cd3d9a5c0 [SPARK-7930] [CORE] [STREAMING] Fixed shutdown hook priorities Shutdown hook for temp directories had priority 100 while SparkContext was 50. So the local root directory was deleted before SparkContext was shutdown.

spark git commit: [HOTFIX] Minor style fix from last commit

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/master e714ecf27 - 36067ce39 [HOTFIX] Minor style fix from last commit Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/36067ce3 Tree:

[1/2] spark git commit: [SPARK-7931] [STREAMING] Do not restart receiver when stopped

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 68559423a - e419821c3 [SPARK-7931] [STREAMING] Do not restart receiver when stopped Attempts to restart the socket receiver when it is supposed to be stopped causes undesirable error messages. Author: Tathagata Das

spark git commit: [MINOR] fix RegressionEvaluator doc

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/master c45d58c14 - 834e69952 [MINOR] fix RegressionEvaluator doc `make clean html` under `python/doc` returns ~~~ /Users/meng/src/spark/python/pyspark/ml/evaluation.py:docstring of pyspark.ml.evaluation.RegressionEvaluator.setParams:3: WARNING:

spark git commit: [MINOR] fix RegressionEvaluator doc

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 6e99dd5d0 - 1d49d8c3f [MINOR] fix RegressionEvaluator doc `make clean html` under `python/doc` returns ~~~ /Users/meng/src/spark/python/pyspark/ml/evaluation.py:docstring of pyspark.ml.evaluation.RegressionEvaluator.setParams:3:

spark git commit: [SPARK-7926] [PYSPARK] use the official Pyrolite release

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/master b069ad23d - c45d58c14 [SPARK-7926] [PYSPARK] use the official Pyrolite release Switch to the official Pyrolite release from the one published under `org.spark-project`. Thanks irmen for making the releases on Maven Central. We didn't

spark git commit: [SPARK-7926] [PYSPARK] use the official Pyrolite release

2015-05-28 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.4 b3a590061 - 6e99dd5d0 [SPARK-7926] [PYSPARK] use the official Pyrolite release Switch to the official Pyrolite release from the one published under `org.spark-project`. Thanks irmen for making the releases on Maven Central. We didn't

spark git commit: [SPARK-7930] [CORE] [STREAMING] Fixed shutdown hook priorities

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 aee046dfa - f7cb272b7 [SPARK-7930] [CORE] [STREAMING] Fixed shutdown hook priorities Shutdown hook for temp directories had priority 100 while SparkContext was 50. So the local root directory was deleted before SparkContext was

spark git commit: [SPARK-7931] [STREAMING] Do not restart receiver when stopped

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/master db9513789 - e714ecf27 [SPARK-7931] [STREAMING] Do not restart receiver when stopped Attempts to restart the socket receiver when it is supposed to be stopped causes undesirable error messages. Author: Tathagata Das

[2/2] spark git commit: [HOTFIX] Minor style fix from last commit

2015-05-28 Thread pwendell
[HOTFIX] Minor style fix from last commit Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e419821c Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e419821c Diff:

[1/2] spark git commit: Preparing Spark release v1.4.0-rc3

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 e419821c3 - 119c93af9 Preparing Spark release v1.4.0-rc3 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2d97d7a0 Tree:

spark git commit: [SPARK-7782] fixed sort arrow issue

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/master 35410614d - e838a25bd [SPARK-7782] fixed sort arrow issue Current behaviour:: In spark UI ![screen shot 2015-05-27 at 3 27 51 pm](https://cloud.githubusercontent.com/assets/3919211/7837541/47d330ba-04a5-11e5-89d1-e5b11da1a513.png) In

spark git commit: [SPARK-7782] fixed sort arrow issue

2015-05-28 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.4 63be026da - bd568df22 [SPARK-7782] fixed sort arrow issue Current behaviour:: In spark UI ![screen shot 2015-05-27 at 3 27 51 pm](https://cloud.githubusercontent.com/assets/3919211/7837541/47d330ba-04a5-11e5-89d1-e5b11da1a513.png) In

Git Push Summary

2015-05-28 Thread pwendell
Repository: spark Updated Tags: refs/tags/v1.4.0-rc3 [deleted] 4983dfc87 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-7895] [STREAMING] [EXAMPLES] Move Kafka examples from scala-2.10/src to src

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-1.4 bd568df22 - ab62d73dd [SPARK-7895] [STREAMING] [EXAMPLES] Move Kafka examples from scala-2.10/src to src Since `spark-streaming-kafka` now is published for both Scala 2.10 and 2.11, we can move `KafkaWordCount` and

spark git commit: [SPARK-7895] [STREAMING] [EXAMPLES] Move Kafka examples from scala-2.10/src to src

2015-05-28 Thread pwendell
Repository: spark Updated Branches: refs/heads/master e838a25bd - 000df2f0d [SPARK-7895] [STREAMING] [EXAMPLES] Move Kafka examples from scala-2.10/src to src Since `spark-streaming-kafka` now is published for both Scala 2.10 and 2.11, we can move `KafkaWordCount` and `DirectKafkaWordCount`