spark git commit: [SPARK-21273][SQL] Propagate logical plan stats using visitor pattern and mixin

2017-06-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 61b5df567 -> b1d719e7c [SPARK-21273][SQL] Propagate logical plan stats using visitor pattern and mixin ## What changes were proposed in this pull request? We currently implement statistics propagation directly in logical plan. Given we

spark git commit: [SPARK-21127][SQL] Update statistics after data changing commands

2017-06-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 4eb41879c -> 61b5df567 [SPARK-21127][SQL] Update statistics after data changing commands ## What changes were proposed in this pull request? Update stats after the following data changing commands: - InsertIntoHadoopFsRelationCommand -

spark git commit: [SPARK-17528][SQL] data should be copied properly before saving into InternalRow

2017-06-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master fd1325522 -> 4eb41879c [SPARK-17528][SQL] data should be copied properly before saving into InternalRow ## What changes were proposed in this pull request? For performance reasons, `UnsafeRow.getString`, `getStruct`, etc. return a

[spark] Git Push Summary

2017-06-30 Thread pwendell
Repository: spark Updated Tags: refs/tags/v2.2.0-rc6 [created] a2c7b2133 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

[2/2] spark git commit: Preparing development version 2.2.1-SNAPSHOT

2017-06-30 Thread pwendell
Preparing development version 2.2.1-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/85fddf40 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/85fddf40 Diff:

[1/2] spark git commit: Preparing Spark release v2.2.0-rc6

2017-06-30 Thread pwendell
Repository: spark Updated Branches: refs/heads/branch-2.2 29a0be2b3 -> 85fddf406 Preparing Spark release v2.2.0-rc6 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/a2c7b213 Tree:

spark git commit: [SPARK-21052][SQL][FOLLOW-UP] Add hash map metrics to join

2017-06-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master eed9c4ef8 -> fd1325522 [SPARK-21052][SQL][FOLLOW-UP] Add hash map metrics to join ## What changes were proposed in this pull request? Remove `numHashCollisions` in `BytesToBytesMap`. And change `getAverageProbesPerLookup()` to

spark git commit: [SPARK-21129][SQL] Arguments of SQL function call should not be named expressions

2017-06-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/branch-2.2 8b08fd06c -> 29a0be2b3 [SPARK-21129][SQL] Arguments of SQL function call should not be named expressions ### What changes were proposed in this pull request? Function argument should not be named expressions. It could cause two

spark git commit: [SPARK-21129][SQL] Arguments of SQL function call should not be named expressions

2017-06-30 Thread lixiao
Repository: spark Updated Branches: refs/heads/master 1fe08d62f -> eed9c4ef8 [SPARK-21129][SQL] Arguments of SQL function call should not be named expressions ### What changes were proposed in this pull request? Function argument should not be named expressions. It could cause two issues: -

spark git commit: [SPARK-21223] Change fileToAppInfo in FsHistoryProvider to fix concurrent issue.

2017-06-30 Thread srowen
Repository: spark Updated Branches: refs/heads/master 528c9281a -> 1fe08d62f [SPARK-21223] Change fileToAppInfo in FsHistoryProvider to fix concurrent issue. # What issue does this PR address ? Jira:https://issues.apache.org/jira/browse/SPARK-21223 fix the Thread-safety issue in

spark git commit: [ML] Fix scala-2.10 build failure of GeneralizedLinearRegressionSuite.

2017-06-30 Thread yliang
Repository: spark Updated Branches: refs/heads/master 3c2fc19d4 -> 528c9281a [ML] Fix scala-2.10 build failure of GeneralizedLinearRegressionSuite. ## What changes were proposed in this pull request? Fix scala-2.10 build failure of ```GeneralizedLinearRegressionSuite```. ## How was this

spark git commit: [SPARK-18294][CORE] Implement commit protocol to support `mapred` package's committer

2017-06-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 49d767d83 -> 3c2fc19d4 [SPARK-18294][CORE] Implement commit protocol to support `mapred` package's committer ## What changes were proposed in this pull request? This PR makes the following changes: - Implement a new commit protocol

spark git commit: [SPARK-18710][ML] Add offset in GLM

2017-06-30 Thread yliang
Repository: spark Updated Branches: refs/heads/master 52981715b -> 49d767d83 [SPARK-18710][ML] Add offset in GLM ## What changes were proposed in this pull request? Add support for offset in GLM. This is useful for at least two reasons: 1. Account for exposure: e.g., when modeling the number

spark git commit: Revert "[SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling"

2017-06-30 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.1 d995dac1c -> 3ecef2491 Revert "[SPARK-21258][SQL] Fix WindowExec complex object aggregation with spilling" This reverts commit d995dac1cdeec940364453675f59ce5cf2b53684. Project: http://git-wip-us.apache.org/repos/asf/spark/repo

spark git commit: [SPARK-20889][SPARKR] Grouped documentation for COLLECTION column methods

2017-06-30 Thread felixcheung
Repository: spark Updated Branches: refs/heads/master fddb63f46 -> 52981715b [SPARK-20889][SPARKR] Grouped documentation for COLLECTION column methods ## What changes were proposed in this pull request? Grouped documentation for column collection methods. Author: actuaryzhang