spark git commit: [SPARK-18823][SPARKR] add support for assigning to column

2017-01-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/branch-2.1 570e5e11d -> 9c04e427d [SPARK-18823][SPARKR] add support for assigning to column ## What changes were proposed in this pull request? Support for ``` df[[myname]] <- 1 df[[2]] <- df$eruptions ``` ## How was this patch tested? manual t

spark git commit: [SPARK-18823][SPARKR] add support for assigning to column

2017-01-24 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master ec9493b44 -> f27e02476 [SPARK-18823][SPARKR] add support for assigning to column ## What changes were proposed in this pull request? Support for ``` df[[myname]] <- 1 df[[2]] <- df$eruptions ``` ## How was this patch tested? manual tests

spark git commit: [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm failing in edge case

2017-01-24 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.1 9c04e427d -> d128b6a39 [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm failing in edge case [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm failing in edge case where no children exist in updateAssignments ## What changes were p

spark git commit: [DOCS] Fix typo in docs

2017-01-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master f27e02476 -> 7c61c2a1c [DOCS] Fix typo in docs ## What changes were proposed in this pull request? Fix typo in docs ## How was this patch tested? Author: uncleGen Closes #16658 from uncleGen/typo-issue. Project: http://git-wip-us.apa

spark git commit: delete useless var “j”

2017-01-24 Thread srowen
Repository: spark Updated Branches: refs/heads/master 7c61c2a1c -> cca868004 delete useless var “j” the var “j” defined in "var j = 0" is useless for “def compress” Author: Souljoy Zhuo Closes #16676 from xiaoyesoso/patch-1. Project: http://git-wip-us.apache.org/repos/asf/spar

spark git commit: [SPARK-19152][SQL] DataFrameWriter.saveAsTable support hive append

2017-01-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/master cca868004 -> 3c86fdddf [SPARK-19152][SQL] DataFrameWriter.saveAsTable support hive append ## What changes were proposed in this pull request? After [SPARK-19107](https://issues.apache.org/jira/browse/SPARK-19107), we now can treat hive as

spark git commit: [SPARK-19246][SQL] CataLogTable's partitionSchema order and exist check

2017-01-24 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 3c86fdddf -> 752502be0 [SPARK-19246][SQL] CataLogTable's partitionSchema order and exist check ## What changes were proposed in this pull request? CataLogTable's partitionSchema should check if each column name in partitionColumnNames mus

spark git commit: [SPARK-14049][CORE] Add functionality in spark history sever API to query applications by end time

2017-01-24 Thread irashid
Repository: spark Updated Branches: refs/heads/master 752502be0 -> 0ff67a1cf [SPARK-14049][CORE] Add functionality in spark history sever API to query applications by end time ## What changes were proposed in this pull request? Currently, spark history server REST API provides functionality

spark git commit: [SPARK-17913][SQL] compare atomic and string type column may return confusing result

2017-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 0ff67a1cf -> 59c184e02 [SPARK-17913][SQL] compare atomic and string type column may return confusing result ## What changes were proposed in this pull request? Spark SQL follows MySQL to do the implicit type conversion for binary compari

spark git commit: [SPARK-18036][ML][MLLIB] Fixing decision trees handling edge cases

2017-01-24 Thread jkbradley
Repository: spark Updated Branches: refs/heads/master 59c184e02 -> d9783380f [SPARK-18036][ML][MLLIB] Fixing decision trees handling edge cases ## What changes were proposed in this pull request? Decision trees/GBT/RF do not handle edge cases such as constant features or empty features. In t

[1/2] spark git commit: [SPARK-19139][CORE] New auth mechanism for transport library.

2017-01-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master d9783380f -> 8f3f73abc http://git-wip-us.apache.org/repos/asf/spark/blob/8f3f73ab/common/network-common/src/test/java/org/apache/spark/network/crypto/AuthIntegrationSuite.java -

[2/2] spark git commit: [SPARK-19139][CORE] New auth mechanism for transport library.

2017-01-24 Thread zsxwing
[SPARK-19139][CORE] New auth mechanism for transport library. This change introduces a new auth mechanism to the transport library, to be used when users enable strong encryption. This auth mechanism has better security than the currently used DIGEST-MD5. The new protocol uses symmetric key encry

spark git commit: [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results

2017-01-24 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.1 d128b6a39 -> b94fb284b [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results ## What changes were proposed in this pull request? This PR fixes the code in Optimizer phase where the NULL-aware expres

spark git commit: [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results

2017-01-24 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master 8f3f73abc -> cdb691eb4 [SPARK-19017][SQL] NOT IN subquery with more than one column may return incorrect results ## What changes were proposed in this pull request? This PR fixes the code in Optimizer phase where the NULL-aware expression

spark git commit: [SPARK-19334][SQL] Fix the code injection vulnerability related to Generator functions.

2017-01-24 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/master cdb691eb4 -> 15ef3740d [SPARK-19334][SQL] Fix the code injection vulnerability related to Generator functions. ## What changes were proposed in this pull request? Similar to SPARK-15165, codegen is in danger of arbitrary code injection. T

spark git commit: [SPARK-19330][DSTREAMS] Also show tooltip for successful batches

2017-01-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master 15ef3740d -> 40a4cfc7c [SPARK-19330][DSTREAMS] Also show tooltip for successful batches ## What changes were proposed in this pull request? ### Before ![_streaming_before](https://cloud.githubusercontent.com/assets/15843379/22181462/1e45c2

spark git commit: [SPARK-19330][DSTREAMS] Also show tooltip for successful batches

2017-01-24 Thread zsxwing
Repository: spark Updated Branches: refs/heads/branch-2.1 b94fb284b -> c13378796 [SPARK-19330][DSTREAMS] Also show tooltip for successful batches ## What changes were proposed in this pull request? ### Before ![_streaming_before](https://cloud.githubusercontent.com/assets/15843379/22181462/1e

spark git commit: [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide

2017-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 40a4cfc7c -> 3fdce8143 [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide ## What changes were proposed in this pull request? - A separate subsection for Aggregations under “Getting Started” in the Spark SQL programmi

spark git commit: [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide

2017-01-24 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.1 c13378796 -> e2f773923 [SPARK-16046][DOCS] Aggregations in the Spark SQL programming guide ## What changes were proposed in this pull request? - A separate subsection for Aggregations under “Getting Started” in the Spark SQL progr