[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89652140 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89647893 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89654874 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89645958 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89647985 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89646058 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89654611 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89652990 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89655174 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -356,10 +356,17 @@ case class

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89656204 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89655153 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala --- @@ -356,10 +356,17 @@ case class

[GitHub] spark issue #15977: [SPARK-18436][SQL] isin causing SQL syntax error with JD...

2016-11-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15977 LGTM. Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89672415 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,326

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89672476 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,326

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89672494 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,326

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89672499 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,326

[GitHub] spark pull request #14136: [SPARK-16282][SQL] Implement percentile SQL funct...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/14136#discussion_r89672530 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -0,0 +1,292

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16015 LGTM pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16015: [SPARK-17251][SQL] Improve `OuterReference` to be `Named...

2016-11-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16015 Merging to master/2.1/2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16012: [SPARK-17251][SQL] Support `OuterReference` in pr...

2016-11-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16012#discussion_r89699139 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -989,7 +989,7 @@ class Analyzer

[GitHub] spark pull request #16026: [SPARK-18597][SQL] Do push-down predicates to rig...

2016-11-27 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16026 [SPARK-18597][SQL] Do push-down predicates to right side for a LEFT ANTI join ## What changes were proposed in this pull request? We currently push down join predicates for a Left Anti join

[GitHub] spark issue #16026: [SPARK-18597][SQL] Do push-down predicates to right side...

2016-11-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16026 cc @dongjoon-hyun could you take a look at this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #16027: Make sure CollapseWindow returns the attributes i...

2016-11-27 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16027 Make sure CollapseWindow returns the attributes in the same order. ## What changes were proposed in this pull request? The `CollapseWindow` optimizer rule changes the order of output

[GitHub] spark issue #16027: Make sure CollapseWindow returns the attributes in the s...

2016-11-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16027 cc @dongjoon-hyun --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #16027: [SPARK-18604][SQL] Make sure CollapseWindow returns the ...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16027 Merging to master/2.1. Thanks for the review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #16034: [SPARK-18527][SQL] Convert decimal array to doubl...

2016-11-28 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16034 [SPARK-18527][SQL] Convert decimal array to double array double for Hive UDAFPercentile ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix

[GitHub] spark issue #16032: [SPARK-18118][SQL] fix a compilation error due to nested...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16032 LGTM - merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16032: [SPARK-18118][SQL] fix a compilation error due to...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16032#discussion_r89772748 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -896,19 +896,25 @@ case class

[GitHub] spark issue #16036: [SPARK-17732][SQL] Revert ALTER TABLE DROP PARTITION sho...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16036 LGTM pending jenkins. @dongjoon-hyun lets merge the implementation of `ALTER TABLE DROP PARTITION` with your fix for it in https://github.com/apache/spark/pull/15987. --- If your

[GitHub] spark pull request #15958: [SPARK-17932][SQL] Support SHOW TABLES EXTENDED L...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15958#discussion_r89794720 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala --- @@ -589,30 +589,49 @@ case class DescribeTableCommand

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15358 LGTM. Merging to master/2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15358: [SPARK-17783] [SQL] Hide Credentials in CREATE and DESC ...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15358 @gatorsmile I cannot merge this into branch 2.0. Could you open a PR if you feel that we should have this in 2.0? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #16026: [SPARK-18597][SQL] Do not push-down join conditions to t...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16026 I am merging this to master/2.1. Thanks for the reviews! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16039: [SPARK-18597][SQL] Do not push-down join conditio...

2016-11-28 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16039 [SPARK-18597][SQL] Do not push-down join conditions to the left side of a Left Anti join [BRANCH-2.0] ## What changes were proposed in this pull request? We currently push down join

[GitHub] spark issue #15722: [SPARK-18208] [Shuffle] Executor OOM due to a growing Lo...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15722 ping @jiexiong --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #16032: [SPARK-18118][SQL] fix a compilation error due to...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16032#discussion_r89814293 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -896,19 +896,25 @@ case class

[GitHub] spark pull request #16041: [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResu...

2016-11-28 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16041 [SPARK-18058][SQL][TRIVIAL] Use dataType.sameResult(...) instead equality on asNullable datatypes ## What changes were proposed in this pull request? This is absolutely minor. PR https

[GitHub] spark issue #16036: [SPARK-17732][SQL] Revert ALTER TABLE DROP PARTITION sho...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16036 LGTM. Merging to master (2.1 was already reverted). Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #16039: [SPARK-18597][SQL] Do not push-down join conditions to t...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16039 I am merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 LGTM. Merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16039: [SPARK-18597][SQL] Do not push-down join conditio...

2016-11-28 Thread hvanhovell
Github user hvanhovell closed the pull request at: https://github.com/apache/spark/pull/16039 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 @rxin this is a very contained patch. It only adds the percentile function. The advantage here is that this further reduces our dependency on relatively slow Hive UDAFs (one more to go), and

[GitHub] spark issue #16043: [SPARK-18601][SQL] Simplify Create/Get complex expressio...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16043 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89883785 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/Spark18601Suite.scala --- @@ -0,0 +1,331 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89883711 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/Spark18601Suite.scala --- @@ -0,0 +1,331 @@ +/* --- End diff -- Move this into

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89883383 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypeConstructors.scala --- @@ -0,0 +1,79 @@ +/* --- End

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89883879 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/Spark18601Suite.scala --- @@ -0,0 +1,331 @@ +/* --- End diff -- See the tests in

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89886872 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypeConstructors.scala --- @@ -0,0 +1,79

[GitHub] spark pull request #16043: [SPARK-18601][SQL] Simplify Create/Get complex ex...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16043#discussion_r89887374 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/Spark18601Suite.scala --- @@ -0,0 +1,331 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #16034: [SPARK-18527][SQL] Convert decimal array to double array...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16034 We have merged the percentile PR. So this does not need to be added to branch-2.1. branch-2.0 still has this problems, should we add it there? cc @rxin @yhuai --- If your project is set up

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 @nsyca this was not uncovered because we didn't add tests, could you add one? `org.apache.spark.sql.catalyst.optimizer.FilterPushdownSuite` seems a good place (they could be similar to the

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 I know it is currently not exposed due to the way we optimize plans. However this may change, and it is better to be safe than sorry. Also note that we typically only test the rule at hand in

[GitHub] spark issue #16034: [SPARK-18527][SQL] Convert decimal array to double array...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16034 closing :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16034: [SPARK-18527][SQL] Convert decimal array to doubl...

2016-11-28 Thread hvanhovell
Github user hvanhovell closed the pull request at: https://github.com/apache/spark/pull/16034 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 You can just write an optimizer test case. You can for example add this to the `FilterPushdownSuite`: ```scala test("joins: only push down join conditions to the right of an exis

[GitHub] spark pull request #16044: [Spark-18614][SQL] Incorrect predicate pushdown f...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16044#discussion_r89962468 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JoinSuite.scala --- @@ -575,6 +575,24 @@ class JoinSuite extends QueryTest with SharedSQLContext

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89984727 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1011,24 +1011,24 @@ class Analyzer( private

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89975192 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1110,6 +1157,12 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89977826 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89978030 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89974976 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1077,10 +1077,54 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89980037 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r89979376 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark issue #16047: [SPARK-17783] [SQL] [BACKPORT-2.0] Hide Credentials in C...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16047 LGTM - merging to 2.0. Thanks! Can you close? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r90043228 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark pull request #16057: [SPARK-18624][SQL] Implicit cast complex types

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16057#discussion_r90011683 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -714,6 +714,17 @@ object TypeCoercion

[GitHub] spark pull request #16057: [SPARK-18624][SQL] Implicit cast complex types

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16057#discussion_r90012747 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -714,6 +714,17 @@ object TypeCoercion

[GitHub] spark pull request #16057: [SPARK-18624][SQL] Implicit cast complex types

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16057#discussion_r8343 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -714,6 +714,17 @@ object TypeCoercion

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r90064101 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark pull request #16046: [SPARK-18582][SQL] Whitelist LogicalPlan operator...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16046#discussion_r90065066 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1120,47 +1173,54 @@ class Analyzer

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16063: [SPARK-18622][SQL] Remove TypeCoercion rules for ...

2016-11-29 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16063 [SPARK-18622][SQL] Remove TypeCoercion rules for Average and Sum aggregate functions ## What changes were proposed in this pull request? Spark currently has special analyzer rules for the

[GitHub] spark issue #16063: [SPARK-18622][SQL] Remove TypeCoercion rules for Average...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16063 cc @gatorsmile @cloud-fan could you take a look at this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #16066: [SPARK-18632][SQL] AggregateFunction should not i...

2016-11-29 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/16066 [SPARK-18632][SQL] AggregateFunction should not implement ImplicitCastInputTypes ## What changes were proposed in this pull request? `AggregateFunction` currently implements

[GitHub] spark issue #16066: [SPARK-18632][SQL] AggregateFunction should not implemen...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16066 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #16044: [Spark-18614][SQL] Incorrect predicate pushdown from Exi...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/16044 LGTM - merging to master/2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16063: [SPARK-18622][SQL] Remove TypeCoercion rules for ...

2016-11-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/16063#discussion_r90140461 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -482,21 +482,6 @@ object TypeCoercion

[GitHub] spark pull request #15484: [SPARK-17868][SQL] Do not use bitmasks during par...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15484#discussion_r84577307 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -518,59 +517,58 @@ case class

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14788 I prefer option 1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15072 @HyukjinKwon can we move this into a set specific method within Dataset itself, for example: ```scala /** A convenient function to wrap a set based logical plan and produce a Dataset

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 +1 on adding a test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15590: [SPARK-17949][SQL] A JVM object based aggregate operator

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15590 This is pretty cool. I was wondering how much code this shares with the regular hash based aggregation path? And if it is worth the effort to integrate the two paths? We can do this in

[GitHub] spark pull request #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15596#discussion_r84579127 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/limit.scala --- @@ -36,14 +36,14 @@ import org.apache.spark.util.Utils case class

[GitHub] spark pull request #15072: [SPARK-17123][SQL] Use type-widened encoder for D...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/15072#discussion_r84579406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2725,4 +2725,14 @@ class Dataset[T] private[sql]( @inline private def

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15072 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15072 Merging to master and 2.0. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15072: [SPARK-17123][SQL] Use type-widened encoder for DataFram...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15072 @HyukjinKwon I cannot cherry-pick this into 2.0. Can you open a PR against 2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #14788: [SPARK-17174][SQL] Add the support for TimestampType for...

2016-10-22 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14788 It would work with different names. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15575: [SPARK-18038] [SQL] Move output partitioning definition ...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15575 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15582: [SPARK-18045][SQL][TESTS] Move `HiveDataFrameAnalyticsSu...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15582 Merging to master. Thanks! Ping me for the follow-up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15601: [SPARK-17123][SQL][BRANCH-2.0] Use type-widened encoder ...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15601 LGTM. Merging to branch-2.0. Could you close this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14136: [SPARK-16282][SQL] Implement percentile SQL function.

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/14136 It is relatively straight forward to test `ImperativeAggregate`s. See: - org.apache.spark.sql.catalyst.expressions.aggregate.HyperLogLogPlusPlusSuite

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 @CodingCat could you open a backport for branch-2.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15595: [SPARK-18058][SQL] Comparing column types ignoring Nulla...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15595 Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark issue #15596: [SQL] Remove shuffle codes in CollectLimitExec

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15596 @JoshRosen could you shed some light on why we are not using the regular `EnsureRequirements` based code path for `CollectLimitExec`? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15602 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15602: [SPARK-18058][SQL] [BRANCH-2.0]Comparing column types ig...

2016-10-23 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15602 Merging to branch-2.0. Thanks! Can you close this PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #15605: [WIP] [SPARK-18067] [SQL] SortMergeJoin adds shuffle if ...

2016-10-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15605 @tejasapatil this didn't work? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature en

[GitHub] spark issue #15609: [SPARK-18048][SQL] To make behaviour of If consistent, i...

2016-10-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15609 @priyankagargnitk this should be covered in type widening. The `trueValue` and `falseValue` expressions should have the same type by the time we reach the execution phase. The `IfCoercion` rule

<    3   4   5   6   7   8   9   10   11   12   >