spark git commit: [SPARK-15312][SQL] Detect Duplicate Key in Partition Spec and Table Properties

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 da5d2300e -> 199bac8fa [SPARK-15312][SQL] Detect Duplicate Key in Partition Spec and Table Properties What changes were proposed in this pull request? When there are duplicate keys in the partition specs or table properties, we al

spark git commit: [SPARK-15312][SQL] Detect Duplicate Key in Partition Spec and Table Properties

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 6d0bfb960 -> a11175eec [SPARK-15312][SQL] Detect Duplicate Key in Partition Spec and Table Properties What changes were proposed in this pull request? When there are duplicate keys in the partition specs or table properties, we always

spark git commit: Small documentation and style fix.

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 6cb8f836d -> 6d0bfb960 Small documentation and style fix. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/6d0bfb96 Tree: http://git-wip-us.apache.org/repos/asf/spark/tr

spark git commit: [SPARK-15396][SQL][DOC] It can't connect hive metastore database

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 fd7e83119 -> da5d2300e [SPARK-15396][SQL][DOC] It can't connect hive metastore database What changes were proposed in this pull request? The `hive.metastore.warehouse.dir` property in hive-site.xml is deprecated since Spark 2.0.0.

spark git commit: [SPARK-15396][SQL][DOC] It can't connect hive metastore database

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 223f63390 -> 6cb8f836d [SPARK-15396][SQL][DOC] It can't connect hive metastore database What changes were proposed in this pull request? The `hive.metastore.warehouse.dir` property in hive-site.xml is deprecated since Spark 2.0.0. Use

spark git commit: [SPARK-15415][SQL] Fix BroadcastHint when autoBroadcastJoinThreshold is 0 or -1

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 9a08c9f1c -> fd7e83119 [SPARK-15415][SQL] Fix BroadcastHint when autoBroadcastJoinThreshold is 0 or -1 ## What changes were proposed in this pull request? This PR makes BroadcastHint more deterministic by using a special isBroadcastab

spark git commit: [SPARK-15415][SQL] Fix BroadcastHint when autoBroadcastJoinThreshold is 0 or -1

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master df9adb5ec -> 223f63390 [SPARK-15415][SQL] Fix BroadcastHint when autoBroadcastJoinThreshold is 0 or -1 ## What changes were proposed in this pull request? This PR makes BroadcastHint more deterministic by using a special isBroadcastable p

spark git commit: [SPARK-15206][SQL] add testcases for distinct aggregate in having clause

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 9c20c7a33 -> 9a08c9f1c [SPARK-15206][SQL] add testcases for distinct aggregate in having clause ## What changes were proposed in this pull request? Add new test cases for including distinct aggregate in having clause in 2.0 branch. Thi

spark git commit: [SPARK-15206][SQL] add testcases for distinct aggregate in having clause

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 8f0a3d5bc -> df9adb5ec [SPARK-15206][SQL] add testcases for distinct aggregate in having clause ## What changes were proposed in this pull request? Add new test cases for including distinct aggregate in having clause in 2.0 branch. This is

spark git commit: [SPARK-15330][SQL] Implement Reset Command

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 6871deb93 -> 9c20c7a33 [SPARK-15330][SQL] Implement Reset Command What changes were proposed in this pull request? Like `Set` Command in Hive, `Reset` is also supported by Hive. See the link: https://cwiki.apache.org/confluence/di

spark git commit: [SPARK-15330][SQL] Implement Reset Command

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master c18fa464f -> 8f0a3d5bc [SPARK-15330][SQL] Implement Reset Command What changes were proposed in this pull request? Like `Set` Command in Hive, `Reset` is also supported by Hive. See the link: https://cwiki.apache.org/confluence/displa

spark git commit: [SPARK-15280] Input/Output] Refactored OrcOutputWriter and moved serialization to a new class.

2016-05-21 Thread yhuai
Repository: spark Updated Branches: refs/heads/master 201a51f36 -> c18fa464f [SPARK-15280] Input/Output] Refactored OrcOutputWriter and moved serialization to a new class. ## What changes were proposed in this pull request? Refactoring: Separated ORC serialization logic from OrcOutputWriter a

spark git commit: [SPARK-15280] Input/Output] Refactored OrcOutputWriter and moved serialization to a new class.

2016-05-21 Thread yhuai
Repository: spark Updated Branches: refs/heads/branch-2.0 4148a9c2c -> 6871deb93 [SPARK-15280] Input/Output] Refactored OrcOutputWriter and moved serialization to a new class. ## What changes were proposed in this pull request? Refactoring: Separated ORC serialization logic from OrcOutputWrit

spark git commit: [SPARK-15452][SQL] Mark aggregator API as experimental

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/master 5e1ee2898 -> 201a51f36 [SPARK-15452][SQL] Mark aggregator API as experimental ## What changes were proposed in this pull request? The Aggregator API was introduced in 2.0 for Dataset. All typed Dataset APIs should still be marked as experi

spark git commit: [SPARK-15452][SQL] Mark aggregator API as experimental

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 d7bf318a0 -> 4148a9c2c [SPARK-15452][SQL] Mark aggregator API as experimental ## What changes were proposed in this pull request? The Aggregator API was introduced in 2.0 for Dataset. All typed Dataset APIs should still be marked as ex

[3/3] spark git commit: [SPARK-15078] [SQL] Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-21 Thread rxin
[SPARK-15078] [SQL] Add all TPCDS 1.4 benchmark queries for SparkSQL Now that SparkSQL supports all TPC-DS queries, this patch adds all 99 benchmark queries inside SparkSQL. Benchmark only Author: Sameer Agarwal Closes #13188 from sameeragarwal/tpcds-all. (cherry picked from commit a78d6ce37

[2/3] spark git commit: [SPARK-15078] [SQL] Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-21 Thread rxin
http://git-wip-us.apache.org/repos/asf/spark/blob/d7bf318a/sql/core/src/test/resources/tpcds/q49.sql -- diff --git a/sql/core/src/test/resources/tpcds/q49.sql b/sql/core/src/test/resources/tpcds/q49.sql new file mode 100755 index

[1/3] spark git commit: [SPARK-15078] [SQL] Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-21 Thread rxin
Repository: spark Updated Branches: refs/heads/branch-2.0 9a8df0c9a -> d7bf318a0 http://git-wip-us.apache.org/repos/asf/spark/blob/d7bf318a/sql/core/src/test/resources/tpcds/q85.sql -- diff --git a/sql/core/src/test/resources/t

spark git commit: [SPARK-15114][SQL] Column name generated by typed aggregate is super verbose

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 69f39552d -> 9a8df0c9a [SPARK-15114][SQL] Column name generated by typed aggregate is super verbose ## What changes were proposed in this pull request? Generate a shorter default alias for `AggregateExpression `, In this PR, aggregate

spark git commit: [SPARK-15114][SQL] Column name generated by typed aggregate is super verbose

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master f39621c99 -> 5e1ee2898 [SPARK-15114][SQL] Column name generated by typed aggregate is super verbose ## What changes were proposed in this pull request? Generate a shorter default alias for `AggregateExpression `, In this PR, aggregate fun

spark git commit: [SPARK-15462][SQL][TEST] unresolved === false` is enough in testcases.

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/branch-2.0 a93f04dd4 -> 69f39552d [SPARK-15462][SQL][TEST] unresolved === false` is enough in testcases. ## What changes were proposed in this pull request? In only `catalyst` module, there exists 8 evaluation test cases on unresolved expression

spark git commit: [SPARK-15462][SQL][TEST] unresolved === false` is enough in testcases.

2016-05-21 Thread wenchen
Repository: spark Updated Branches: refs/heads/master 666bf2e83 -> f39621c99 [SPARK-15462][SQL][TEST] unresolved === false` is enough in testcases. ## What changes were proposed in this pull request? In only `catalyst` module, there exists 8 evaluation test cases on unresolved expressions. B

spark git commit: [SPARK-15445][SQL] Build fails for java 1.7 after adding java.mathBigInteger support

2016-05-21 Thread srowen
Repository: spark Updated Branches: refs/heads/branch-2.0 c0cc921a5 -> a93f04dd4 [SPARK-15445][SQL] Build fails for java 1.7 after adding java.mathBigInteger support ## What changes were proposed in this pull request? Using longValue() and then checking whether the value is in the range for a

spark git commit: [SPARK-15445][SQL] Build fails for java 1.7 after adding java.mathBigInteger support

2016-05-21 Thread srowen
Repository: spark Updated Branches: refs/heads/master 45b7557e6 -> 666bf2e83 [SPARK-15445][SQL] Build fails for java 1.7 after adding java.mathBigInteger support ## What changes were proposed in this pull request? Using longValue() and then checking whether the value is in the range for a lo