spark git commit: [SPARK-16926][SQL] Add unit test to compare table and partition column metadata.

2016-09-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 06e33985c -> f2d6e2ef2 [SPARK-16926][SQL] Add unit test to compare table and partition column metadata. ## What changes were proposed in this pull request? Add unit test for changes made in PR #14515. It makes sure that a newly created ta

spark git commit: [SPARK-16302][SQL] Set the right number of partitions for reading data from a local collection.

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 5bea8757c -> 06e33985c [SPARK-16302][SQL] Set the right number of partitions for reading data from a local collection. follow #13137 This pr sets the right number of partitions when reading data from a local collection. Query 'val df = Se

spark git commit: [SPARK-16619] Add shuffle service metrics entry in monitoring docs

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 03d77af9e -> 5bea8757c [SPARK-16619] Add shuffle service metrics entry in monitoring docs After change [SPARK-16405](https://github.com/apache/spark/pull/14080), we need to update docs by adding shuffle service metrics entry in currently s

spark git commit: [SPARK-16525] [SQL] Enable Row Based HashMap in HashAggregateExec

2016-09-01 Thread davies
Repository: spark Updated Branches: refs/heads/master 15539e54c -> 03d77af9e [SPARK-16525] [SQL] Enable Row Based HashMap in HashAggregateExec ## What changes were proposed in this pull request? This PR is the second step for the following feature: For hash aggregation in Spark SQL, we use a

spark git commit: [SPARK-17355] Workaround for HIVE-14684 / HiveResultSetMetaData.isSigned exception

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/branch-2.0 ac22ab077 -> dd377a522 [SPARK-17355] Workaround for HIVE-14684 / HiveResultSetMetaData.isSigned exception ## What changes were proposed in this pull request? Attempting to use Spark SQL's JDBC data source against the Hive ThriftServer

spark git commit: [SPARK-17355] Workaround for HIVE-14684 / HiveResultSetMetaData.isSigned exception

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master d314677cf -> 15539e54c [SPARK-17355] Workaround for HIVE-14684 / HiveResultSetMetaData.isSigned exception ## What changes were proposed in this pull request? Attempting to use Spark SQL's JDBC data source against the Hive ThriftServer re

spark git commit: [SPARK-16461][SQL] Support partition batch pruning with `<=>` predicate in InMemoryTableScanExec

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master e388bd544 -> d314677cf [SPARK-16461][SQL] Support partition batch pruning with `<=>` predicate in InMemoryTableScanExec ## What changes were proposed in this pull request? It seems `EqualNullSafe` filter was missed for batch pruneing part

spark git commit: [SPARK-16926] [SQL] Remove partition columns from partition metadata.

2016-09-01 Thread davies
Repository: spark Updated Branches: refs/heads/master edb45734f -> 473d78649 [SPARK-16926] [SQL] Remove partition columns from partition metadata. ## What changes were proposed in this pull request? This removes partition columns from column metadata of partitions to match tables. A change

spark git commit: [SPARK-16732][SQL] Remove unused codes in subexpressionEliminationForWholeStageCodegen

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 473d78649 -> e388bd544 [SPARK-16732][SQL] Remove unused codes in subexpressionEliminationForWholeStageCodegen ## What changes were proposed in this pull request? Some codes in subexpressionEliminationForWholeStageCodegen are never used ac

spark git commit: [SPARK-16926] [SQL] Remove partition columns from partition metadata.

2016-09-01 Thread davies
Repository: spark Updated Branches: refs/heads/branch-2.0 13bacd730 -> ac22ab077 [SPARK-16926] [SQL] Remove partition columns from partition metadata. ## What changes were proposed in this pull request? This removes partition columns from column metadata of partitions to match tables. A cha

spark git commit: [SPARK-16533][HOTFIX] Fix compilation on Scala 2.10.

2016-09-01 Thread vanzin
Repository: spark Updated Branches: refs/heads/master 3893e8c57 -> edb45734f [SPARK-16533][HOTFIX] Fix compilation on Scala 2.10. No idea why it was failing (the needed import was there), but this makes things work. Author: Marcelo Vanzin Closes #14925 from vanzin/SPARK-16533. Project: ht

spark git commit: [SPARK-17331][CORE][MLLIB] Avoid allocating 0-length arrays

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master 2be5f8d7e -> 3893e8c57 [SPARK-17331][CORE][MLLIB] Avoid allocating 0-length arrays ## What changes were proposed in this pull request? Avoid allocating some 0-length arrays, esp. in UTF8String, and by using Array.empty in Scala over Array

spark git commit: [SPARK-17263][SQL] Add hexadecimal literal parsing

2016-09-01 Thread joshrosen
Repository: spark Updated Branches: refs/heads/master a0aac4b77 -> 2be5f8d7e [SPARK-17263][SQL] Add hexadecimal literal parsing ## What changes were proposed in this pull request? This PR adds the ability to parse SQL (hexadecimal) binary literals (AKA bit strings). It follows the following s

spark git commit: [SPARK-16533][CORE] resolve deadlocking in driver when executors die

2016-09-01 Thread vanzin
Repository: spark Updated Branches: refs/heads/master adaaffa34 -> a0aac4b77 [SPARK-16533][CORE] resolve deadlocking in driver when executors die ## What changes were proposed in this pull request? This pull request reverts the changes made as a part of #14605, which simply side-steps the dea

spark git commit: [SPARK-17271][SQL] Planner adds un-necessary Sort even if child orde…

2016-09-01 Thread hvanhovell
Repository: spark Updated Branches: refs/heads/branch-2.0 6281b74b6 -> 13bacd730 [SPARK-17271][SQL] Planner adds un-necessary Sort even if child orde… ## What changes were proposed in this pull request? Ports https://github.com/apache/spark/pull/14841 and https://github.com/apache/spark/pu

spark git commit: [SPARK-17271][SQL] Remove redundant `semanticEquals()` from `SortOrder`

2016-09-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 8e740ae44 -> adaaffa34 [SPARK-17271][SQL] Remove redundant `semanticEquals()` from `SortOrder` ## What changes were proposed in this pull request? Removing `semanticEquals()` from `SortOrder` because it can use the `semanticEquals()` prov

spark git commit: [SPARK-17257][SQL] the physical plan of CREATE TABLE or CTAS should take CatalogTable

2016-09-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 1f06a5b6a -> 8e740ae44 [SPARK-17257][SQL] the physical plan of CREATE TABLE or CTAS should take CatalogTable ## What changes were proposed in this pull request? This is kind of a follow-up of https://github.com/apache/spark/pull/14482 . A

spark git commit: [SPARK-17353][SPARK-16943][SPARK-16942][SQL] Fix multiple bugs in CREATE TABLE LIKE command

2016-09-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master dd859f95c -> 1f06a5b6a [SPARK-17353][SPARK-16943][SPARK-16942][SQL] Fix multiple bugs in CREATE TABLE LIKE command ### What changes were proposed in this pull request? The existing `CREATE TABLE LIKE` command has multiple issues: - The ge

spark git commit: fixed typos

2016-09-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master a18c169fd -> dd859f95c fixed typos fixed 2 typos Author: Seigneurin, Alexis (CONT) Closes #14877 from aseigneurin/fix-typo-2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spa

spark git commit: [SPARK-16283][SQL] Implements percentile_approx aggregation function which supports partial aggregation.

2016-09-01 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 536fa911c -> a18c169fd [SPARK-16283][SQL] Implements percentile_approx aggregation function which supports partial aggregation. ## What changes were proposed in this pull request? This PR implements aggregation function `percentile_approx

spark git commit: [SPARK-17329][BUILD] Don't build PRs with -Pyarn unless YARN code changed

2016-09-01 Thread srowen
Repository: spark Updated Branches: refs/heads/master 21c0a4fe9 -> 536fa911c [SPARK-17329][BUILD] Don't build PRs with -Pyarn unless YARN code changed ## What changes were proposed in this pull request? Only build PRs with -Pyarn if YARN code was modified. ## How was this patch tested? Jenk