spark git commit: [SPARK-22306][SQL] alter table schema should not erase the bucketing metadata at hive side

2017-11-02 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 882079f5c -> 2fd12af43 [SPARK-22306][SQL] alter table schema should not erase the bucketing metadata at hive side forward-port https://github.com/apache/spark/pull/19622 to master branch. This bug doesn't exist in master because we've

[spark] Git Push Summary

2017-11-02 Thread holden
Repository: spark Updated Tags: refs/tags/v2.1.2 [created] 2abaea9e4 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-22243][DSTREAM] spark.yarn.jars should reload from config when checkpoint recovery

2017-11-02 Thread zsxwing
Repository: spark Updated Branches: refs/heads/master e3f67a97f -> 882079f5c [SPARK-22243][DSTREAM] spark.yarn.jars should reload from config when checkpoint recovery ## What changes were proposed in this pull request? the previous [PR](https://github.com/apache/spark/pull/19469) is deleted

spark git commit: [SPARK-22416][SQL] Move OrcOptions from `sql/hive` to `sql/core`

2017-11-02 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 41b60125b -> e3f67a97f [SPARK-22416][SQL] Move OrcOptions from `sql/hive` to `sql/core` ## What changes were proposed in this pull request? According to the [discussion](https://github.com/apache/spark/pull/19571#issuecomment-339472976)

spark git commit: [SPARK-22369][PYTHON][DOCS] Exposes catalog API documentation in PySpark

2017-11-02 Thread rxin
Repository: spark Updated Branches: refs/heads/master b2463fad7 -> 41b60125b [SPARK-22369][PYTHON][DOCS] Exposes catalog API documentation in PySpark ## What changes were proposed in this pull request? This PR proposes to add a link from `spark.catalog(..)` to `Catalog` and expose Catalog

spark git commit: [SPARK-22145][MESOS] fix supervise with checkpointing on mesos

2017-11-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master 277b1924b -> b2463fad7 [SPARK-22145][MESOS] fix supervise with checkpointing on mesos ## What changes were proposed in this pull request? - Fixes the issue with the frameworkId being recovered by checkpointed data overwriting the one

spark git commit: [SPARK-22408][SQL] RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages

2017-11-02 Thread rxin
Repository: spark Updated Branches: refs/heads/master 849b465bb -> 277b1924b [SPARK-22408][SQL] RelationalGroupedDataset's distinct pivot value calculation launches unnecessary stages ## What changes were proposed in this pull request? Adding a global limit on top of the distinct values

spark git commit: [SPARK-22306][SQL][2.2] alter table schema should not erase the bucketing metadata at hive side

2017-11-02 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.2 c311c5e79 -> 4074ed2e1 [SPARK-22306][SQL][2.2] alter table schema should not erase the bucketing metadata at hive side ## What changes were proposed in this pull request? When we alter table schema, we set the new schema to spark

spark git commit: [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala 2.12

2017-11-02 Thread srowen
Repository: spark Updated Branches: refs/heads/master b04eefae4 -> 849b465bb [SPARK-14650][REPL][BUILD] Compile Spark REPL for Scala 2.12 ## What changes were proposed in this pull request? Spark REPL changes for Scala 2.12.4: use command(), not processLine() in ILoop; remove direct