svn commit: r1713691 - in /spark: _layouts/ site/ site/docs/ site/graphx/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-11-10 Thread rxin
Author: rxin Date: Tue Nov 10 16:46:33 2015 New Revision: 1713691 URL: http://svn.apache.org/viewvc?rev=1713691=rev Log: update latest doc Modified: spark/_layouts/global.html spark/site/community.html spark/site/docs/latest spark/site/documentation.html

svn commit: r1713690 [2/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-11-10 Thread rxin
Added: spark/site/news/spark-1-5-2-released.html URL: http://svn.apache.org/viewvc/spark/site/news/spark-1-5-2-released.html?rev=1713690=auto == --- spark/site/news/spark-1-5-2-released.html (added) +++

svn commit: r1713690 [3/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-11-10 Thread rxin
Modified: spark/site/releases/spark-release-1-2-1.html URL: http://svn.apache.org/viewvc/spark/site/releases/spark-release-1-2-1.html?rev=1713690=1713689=1713690=diff == --- spark/site/releases/spark-release-1-2-1.html

svn commit: r1713690 [1/3] - in /spark: ./ js/ news/_posts/ releases/_posts/ site/ site/graphx/ site/js/ site/mllib/ site/news/ site/releases/ site/screencasts/ site/sql/ site/streaming/

2015-11-10 Thread rxin
Author: rxin Date: Tue Nov 10 16:44:07 2015 New Revision: 1713690 URL: http://svn.apache.org/viewvc?rev=1713690=rev Log: 1.5.2 release Added: spark/news/_posts/2015-11-09-spark-1-5-2-released.md spark/releases/_posts/2015-11-09-spark-release-1-5-2.md

spark git commit: [SPARK-10863][SPARKR] Method coltypes() (New version)

2015-11-10 Thread shivaram
Repository: spark Updated Branches: refs/heads/branch-1.6 7c4ade0d7 -> d2405cb5e [SPARK-10863][SPARKR] Method coltypes() (New version) This is a follow up on PR #8984, as the corresponding branch for such PR was damaged. Author: Oscar D. Lara Yejas Closes #9579 from

spark git commit: [SPARK-10863][SPARKR] Method coltypes() (New version)

2015-11-10 Thread shivaram
Repository: spark Updated Branches: refs/heads/master e0701c756 -> 47735cdc2 [SPARK-10863][SPARKR] Method coltypes() (New version) This is a follow up on PR #8984, as the corresponding branch for such PR was damaged. Author: Oscar D. Lara Yejas Closes #9579 from

spark git commit: [SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 47735cdc2 -> dfcfcbcc0 [SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation Currently the user facing api for typed aggregation has some limitations: * the customized typed aggregation must be the first of

spark git commit: [SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 d2405cb5e -> 6e2e84f3e [SPARK-11578][SQL][FOLLOW-UP] complete the user facing api for typed aggregation Currently the user facing api for typed aggregation has some limitations: * the customized typed aggregation must be the first of

spark git commit: [SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 53600854c -> 87aedc48c [SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections This patch adds the building blocks for codegening subexpr elimination and implements it end to end for UnsafeProjection. The building blocks

spark git commit: [SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 5ccc1eb08 -> f38509a76 [SPARK-10371][SQL] Implement subexpr elimination for UnsafeProjections This patch adds the building blocks for codegening subexpr elimination and implements it end to end for UnsafeProjection. The building

spark git commit: [SPARK-7841][BUILD] Stop using retrieveManaged to retrieve dependencies in SBT

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 6c1409e0b -> 4dad3c51b [SPARK-7841][BUILD] Stop using retrieveManaged to retrieve dependencies in SBT This patch modifies Spark's SBT build so that it no longer uses `retrieveManaged` / `lib_managed` to store its dependencies. The

spark git commit: [SPARK-11382] Replace example code in mllib-decision-tree.md using include_example

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 39c1ebcd3 -> 6c1409e0b [SPARK-11382] Replace example code in mllib-decision-tree.md using include_example https://issues.apache.org/jira/browse/SPARK-11382 B.T.W. I fix an error in naive_bayes_example.py. Author: Xusen Yin

spark git commit: [SPARK-11590][SQL] use native json_tuple in lateral view

2015-11-10 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.6 6e2e84f3e -> 5ccc1eb08 [SPARK-11590][SQL] use native json_tuple in lateral view Author: Wenchen Fan Closes #9562 from cloud-fan/json-tuple. (cherry picked from commit 53600854c270d4c953fe95fbae528740b5cf6603)

spark git commit: [SPARK-11590][SQL] use native json_tuple in lateral view

2015-11-10 Thread yhuai
Repository: spark Updated Branches: refs/heads/master dfcfcbcc0 -> 53600854c [SPARK-11590][SQL] use native json_tuple in lateral view Author: Wenchen Fan Closes #9562 from cloud-fan/json-tuple. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle

2015-11-10 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 689386b1c -> 6e5fc3788 [SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle with yarn's external shuffle, ExternalShuffleClient of executors reserve its connections

[3/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala -- diff --git

[2/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala -- diff --git

[3/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Utils.scala -- diff --git

[2/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/FilterPushdownSuite.scala -- diff --git

[1/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master 6e5fc3788 -> e0701c756 http://git-wip-us.apache.org/repos/asf/spark/blob/e0701c75/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala -- diff --git

[4/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s https://issues.apache.org/jira/browse/SPARK-9830 This PR contains the following main changes. * Removing `AggregateExpression1`. * Removing `Aggregate` operator, which is used to evaluate

[1/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 825e971d0 -> 7c4ade0d7 http://git-wip-us.apache.org/repos/asf/spark/blob/7c4ade0d/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveQl.scala -- diff --git

[4/4] spark git commit: [SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s

2015-11-10 Thread marmbrus
[SPARK-9830][SQL] Remove AggregateExpression1 and Aggregate Operator used to evaluate AggregateExpression1s https://issues.apache.org/jira/browse/SPARK-9830 This PR contains the following main changes. * Removing `AggregateExpression1`. * Removing `Aggregate` operator, which is used to evaluate

spark git commit: [ML][R] SparkR::glm summary result to compare with native R

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 f38509a76 -> 54685fa36 [ML][R] SparkR::glm summary result to compare with native R Follow up #9561. Due to [SPARK-11587](https://issues.apache.org/jira/browse/SPARK-11587) has been fixed, we should compare SparkR::glm summary result

spark git commit: [SPARK-11618][ML] Minor refactoring of basic ML import/export

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 54685fa36 -> 80641c4fa [SPARK-11618][ML] Minor refactoring of basic ML import/export Refactoring * separated overwrite and param save logic in DefaultParamsWriter * added sparkVersion to DefaultParamsWriter CC: mengxr Author: Joseph

spark git commit: [SPARK-11567] [PYTHON] Add Python API for corr Aggregate function

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.6 6a74efab0 -> 6616f4da3 [SPARK-11567] [PYTHON] Add Python API for corr Aggregate function like `df.agg(corr("col1", "col2")` davies Author: felixcheung Closes #9536 from felixcheung/pyfunc. (cherry picked

spark git commit: [SPARK-11567] [PYTHON] Add Python API for corr Aggregate function

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/master 638c51d93 -> 32790fe72 [SPARK-11567] [PYTHON] Add Python API for corr Aggregate function like `df.agg(corr("col1", "col2")` davies Author: felixcheung Closes #9536 from felixcheung/pyfunc. Project:

spark git commit: [SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and update toString of Exchange

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.6 b79c1bd1e -> ff7d869c4 [SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and update toString of Exchange https://issues.apache.org/jira/browse/SPARK-9830 This is the follow-up pr for

spark git commit: [SPARK-7316][MLLIB] RDD sliding window with step

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 80641c4fa -> f0180106a [SPARK-7316][MLLIB] RDD sliding window with step Implementation of step capability for sliding window function in MLlib's RDD. Though one can use current sliding window with step 1 and then filter every Nth

spark git commit: [SPARK-7316][MLLIB] RDD sliding window with step

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/master 18350a570 -> dba1a62cf [SPARK-7316][MLLIB] RDD sliding window with step Implementation of step capability for sliding window function in MLlib's RDD. Though one can use current sliding window with step 1 and then filter every Nth window,

spark git commit: [SPARK-11616][SQL] Improve toString for Dataset

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/master dba1a62cf -> 724cf7a38 [SPARK-11616][SQL] Improve toString for Dataset Author: Michael Armbrust Closes #9586 from marmbrus/dataset-toString. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-9818] Re-enable Docker tests for JDBC data source

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.6 6616f4da3 -> d0828d224 [SPARK-9818] Re-enable Docker tests for JDBC data source This patch re-enables tests for the Docker JDBC data source. These tests were reverted in #4872 due to transitive dependency conflicts introduced by the

spark git commit: [SPARK-9818] Re-enable Docker tests for JDBC data source

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 32790fe72 -> 1dde39d79 [SPARK-9818] Re-enable Docker tests for JDBC data source This patch re-enables tests for the Docker JDBC data source. These tests were reverted in #4872 due to transitive dependency conflicts introduced by the

spark git commit: [SPARK-5565][ML] LDA wrapper for Pipelines API

2015-11-10 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 1dde39d79 -> e281b8739 [SPARK-5565][ML] LDA wrapper for Pipelines API This adds LDA to spark.ml, the Pipelines API. It follows the design doc in the JIRA: [https://issues.apache.org/jira/browse/SPARK-5565], with one major change: * I

spark git commit: [SPARK-5565][ML] LDA wrapper for Pipelines API

2015-11-10 Thread jkbradley
Repository: spark Updated Branches: refs/heads/branch-1.6 d0828d224 -> b79c1bd1e [SPARK-5565][ML] LDA wrapper for Pipelines API This adds LDA to spark.ml, the Pipelines API. It follows the design doc in the JIRA: [https://issues.apache.org/jira/browse/SPARK-5565], with one major change: * I

spark git commit: [SPARK-11616][SQL] Improve toString for Dataset

2015-11-10 Thread marmbrus
Repository: spark Updated Branches: refs/heads/branch-1.6 f0180106a -> 8fb7b8304 [SPARK-11616][SQL] Improve toString for Dataset Author: Michael Armbrust Closes #9586 from marmbrus/dataset-toString. (cherry picked from commit

spark git commit: [SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and update toString of Exchange

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master e281b8739 -> 3121e7816 [SPARK-9830][SPARK-11641][SQL][FOLLOW-UP] Remove AggregateExpression1 and update toString of Exchange https://issues.apache.org/jira/browse/SPARK-9830 This is the follow-up pr for

spark git commit: [SPARK-11550][DOCS] Replace example code in mllib-optimization.md using include_example

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/master 724cf7a38 -> 638c51d93 [SPARK-11550][DOCS] Replace example code in mllib-optimization.md using include_example Author: Pravin Gadakh Closes #9516 from pravingadakh/SPARK-11550. Project:

spark git commit: [SPARK-11550][DOCS] Replace example code in mllib-optimization.md using include_example

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 8fb7b8304 -> 6a74efab0 [SPARK-11550][DOCS] Replace example code in mllib-optimization.md using include_example Author: Pravin Gadakh Closes #9516 from pravingadakh/SPARK-11550. (cherry picked from commit

spark git commit: [SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3)

2015-11-10 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-1.6 ff7d869c4 -> ce5aba32f [SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3) This PR is a 2nd follow-up for [SPARK-9241](https://issues.apache.org/jira/browse/SPARK-9241). It contains the following improvements: *

spark git commit: [SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in Python

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/master a3989058c -> c0e48dfa6 [SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in Python cc jkbradley Author: Yu ISHIKAWA Closes #9534 from yu-iskw/SPARK-11566. Project:

spark git commit: [SPARK-10192][CORE] simple test w/ failure involving a shared dependency

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master c0e48dfa6 -> 33112f9c4 [SPARK-10192][CORE] simple test w/ failure involving a shared dependency just trying to increase test coverage in the scheduler, this already works. It includes a regression test for SPARK-9809 copied some test

spark git commit: [SPARK-10192][CORE] simple test w/ failure involving a shared dependency

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 d8bfc025c -> 10272d5c9 [SPARK-10192][CORE] simple test w/ failure involving a shared dependency just trying to increase test coverage in the scheduler, this already works. It includes a regression test for SPARK-9809 copied some

spark git commit: [SPARK-11361][STREAMING] Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 900917541 -> 6600786dd [SPARK-11361][STREAMING] Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz Currently, when a DStream sets the scope for RDD generated by it, that scope is not allowed to be

spark git commit: [SPARK-11361][STREAMING] Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 fafeca36e -> b34819c7b [SPARK-11361][STREAMING] Show scopes of RDD operations inside DStream.foreachRDD and DStream.transform in DAG viz Currently, when a DStream sets the scope for RDD generated by it, that scope is not allowed to

spark git commit: [SPARK-11615] Drop @VisibleForTesting annotation

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 93ac30741 -> fafeca36e [SPARK-11615] Drop @VisibleForTesting annotation See http://search-hadoop.com/m/q3RTtjpe8r1iRbTj2 for discussion. Summary: addition of VisibleForTesting annotation resulted in spark-shell malfunctioning.

spark git commit: [SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3)

2015-11-10 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 3121e7816 -> 21c562fa0 [SPARK-9241][SQL] Supporting multiple DISTINCT columns - follow-up (3) This PR is a 2nd follow-up for [SPARK-9241](https://issues.apache.org/jira/browse/SPARK-9241). It contains the following improvements: * Fix

spark git commit: [SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in Python

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.6 9ccd1bb80 -> d8bfc025c [SPARK-11566] [MLLIB] [PYTHON] Refactoring GaussianMixtureModel.gaussians in Python cc jkbradley Author: Yu ISHIKAWA Closes #9534 from yu-iskw/SPARK-11566. (cherry picked from commit

spark git commit: [MINOR] Fix typo in AggregationQuerySuite.scala

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 6600786dd -> 12c7635dc [MINOR] Fix typo in AggregationQuerySuite.scala Author: Forest Fang Closes #9357 from saurfang/patch-1. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit:

spark git commit: [SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/branch-1.6 b34819c7b -> bc3bfa04d [SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel This PR adds model save/load for spark.ml's LogisticRegressionModel. It also does minor refactoring of the default save/load classes to reuse

spark git commit: [SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel

2015-11-10 Thread meng
Repository: spark Updated Branches: refs/heads/master 745e45d5f -> 6e101d2e9 [SPARK-6726][ML] Import/export for spark.ml LogisticRegressionModel This PR adds model save/load for spark.ml's LogisticRegressionModel. It also does minor refactoring of the default save/load classes to reuse

spark git commit: [SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply`

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-1.6 ce5aba32f -> 9ccd1bb80 [SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply` Changed AppClient to be non-blocking in `receiveAndReply` by using a separate thread to wait for response and reply to the

spark git commit: [SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply`

2015-11-10 Thread rxin
Repository: spark Updated Branches: refs/heads/master 21c562fa0 -> a3989058c [SPARK-10827][CORE] AppClient should not use `askWithReply` in `receiveAndReply` Changed AppClient to be non-blocking in `receiveAndReply` by using a separate thread to wait for response and reply to the context.

spark git commit: [SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 33112f9c4 -> 3e0a6cf1e [SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called As vonnagy reported in the following thread: http://search-hadoop.com/m/q3RTtk982kvIow22 Attempts to join the thread in AsynchronousListenerBus

spark git commit: [SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/branch-1.6 10272d5c9 -> 93ac30741 [SPARK-11572] Exit AsynchronousListenerBus thread when stop() is called As vonnagy reported in the following thread: http://search-hadoop.com/m/q3RTtk982kvIow22 Attempts to join the thread in

spark git commit: Fix typo in driver page

2015-11-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.5 0512960fc -> 6e823b4d7 Fix typo in driver page "Comamnd property" => "Command property" Author: Paul Chandler Closes #9578 from pestilence669/fix_spelling. (cherry picked from commit

spark git commit: Fix typo in driver page

2015-11-10 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-1.6 b426d24db -> 39c1ebcd3 Fix typo in driver page "Comamnd property" => "Command property" Author: Paul Chandler Closes #9578 from pestilence669/fix_spelling. (cherry picked from commit

spark git commit: Fix typo in driver page

2015-11-10 Thread srowen
Repository: spark Updated Branches: refs/heads/master 521b3cae1 -> 5507a9d09 Fix typo in driver page "Comamnd property" => "Command property" Author: Paul Chandler Closes #9578 from pestilence669/fix_spelling. Project:

[2/2] spark git commit: [SPARK-11290][STREAMING] Basic implementation of trackStateByKey

2015-11-10 Thread tdas
[SPARK-11290][STREAMING] Basic implementation of trackStateByKey Current updateStateByKey provides stateful processing in Spark Streaming. It allows the user to maintain per-key state and manage that state using an updateFunction. The updateFunction is called for each key, and it uses new data

[1/2] spark git commit: [SPARK-11290][STREAMING] Basic implementation of trackStateByKey

2015-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/branch-1.6 85bc72908 -> daa74be6f http://git-wip-us.apache.org/repos/asf/spark/blob/daa74be6/streaming/src/test/scala/org/apache/spark/streaming/rdd/TrackStateRDDSuite.scala --

[1/2] spark git commit: [SPARK-11290][STREAMING] Basic implementation of trackStateByKey

2015-11-10 Thread tdas
Repository: spark Updated Branches: refs/heads/master bd70244b3 -> 99f5f9886 http://git-wip-us.apache.org/repos/asf/spark/blob/99f5f988/streaming/src/test/scala/org/apache/spark/streaming/rdd/TrackStateRDDSuite.scala -- diff

[2/2] spark git commit: [SPARK-11290][STREAMING] Basic implementation of trackStateByKey

2015-11-10 Thread tdas
[SPARK-11290][STREAMING] Basic implementation of trackStateByKey Current updateStateByKey provides stateful processing in Spark Streaming. It allows the user to maintain per-key state and manage that state using an updateFunction. The updateFunction is called for each key, and it uses new data

spark git commit: [SPARK-11615] Drop @VisibleForTesting annotation

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 3e0a6cf1e -> 900917541 [SPARK-11615] Drop @VisibleForTesting annotation See http://search-hadoop.com/m/q3RTtjpe8r1iRbTj2 for discussion. Summary: addition of VisibleForTesting annotation resulted in spark-shell malfunctioning. Author:

spark git commit: [MINOR] License header formatting fix

2015-11-10 Thread andrewor14
Repository: spark Updated Branches: refs/heads/master 12c7635dc -> 745e45d5f [MINOR] License header formatting fix The header wasn't indented properly. Author: Marc Prud'hommeaux Closes #9312 from mprudhom/patch-1. Project:

spark git commit: [SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402

2015-11-10 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 6e101d2e9 -> fac53d8ec [SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402 This fixes an NPE introduced in SPARK-10192 / #8402. Author: Josh Rosen Closes #9620 from JoshRosen/SPARK-10192-hotfix.

spark git commit: [SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402

2015-11-10 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-1.6 bc3bfa04d -> d9a39048f [SPARK-10192][HOTFIX] Fix NPE in test that was added in #8402 This fixes an NPE introduced in SPARK-10192 / #8402. Author: Josh Rosen Closes #9620 from JoshRosen/SPARK-10192-hotfix.

spark git commit: [SPARK-11468] [SPARKR] add stddev/variance agg functions for Column

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.6 d9a39048f -> 649a0a357 [SPARK-11468] [SPARKR] add stddev/variance agg functions for Column Checked names, none of them should conflict with anything in base shivaram davies rxin Author: felixcheung Closes

spark git commit: [SPARK-11468] [SPARKR] add stddev/variance agg functions for Column

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/master fac53d8ec -> 1a8e0468a [SPARK-11468] [SPARKR] add stddev/variance agg functions for Column Checked names, none of them should conflict with anything in base shivaram davies rxin Author: felixcheung Closes

spark git commit: [SPARK-11463] [PYSPARK] only install signal in main thread

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/branch-1.6 649a0a357 -> 85bc72908 [SPARK-11463] [PYSPARK] only install signal in main thread Only install signal in main thread, or it will fail to create context in not-main thread. Author: Davies Liu Closes #9574 from

spark git commit: [SPARK-11463] [PYSPARK] only install signal in main thread

2015-11-10 Thread davies
Repository: spark Updated Branches: refs/heads/master 1a8e0468a -> bd70244b3 [SPARK-11463] [PYSPARK] only install signal in main thread Only install signal in main thread, or it will fail to create context in not-main thread. Author: Davies Liu Closes #9574 from

Git Push Summary

2015-11-10 Thread rxin
Repository: spark Updated Tags: refs/tags/v1.5.2-rc2 [deleted] 49c30c1f6 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org