[GitHub] spark pull request #17810: [SPARK-20534][SQL] Make outer generate exec retur...

2017-04-30 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17810#discussion_r114077798 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/GeneratorFunctionSuite.scala --- @@ -91,7 +91,7 @@ class GeneratorFunctionSuite extends QueryTest

[GitHub] spark pull request #17810: [SPARK-20534][SQL] Make outer generate exec retur...

2017-04-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17810#discussion_r114066762 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/GeneratorFunctionSuite.scala --- @@ -91,7 +91,7 @@ class GeneratorFunctionSuite extends QueryTest

[GitHub] spark pull request #17810: [SPARK-20534][SQL] Make outer generate exec retur...

2017-04-29 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17810 [SPARK-20534][SQL] Make outer generate exec return empty rows ## What changes were proposed in this pull request? Generate exec does not produce `null` values if the generator for the input

[GitHub] spark issue #17804: [SPARK-20496][SS] Bug in KafkaWriter Looks at Unanalyzed...

2017-04-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17804 @anabranch can you open this against branch-2.1 instead of master? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17596: [SPARK-12837][CORE] Do not send the name of internal acc...

2017-04-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17596 LGTM - merging to master/2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17791: [SPARK-20515][SQL] Fix reading of HIVE ORC table with va...

2017-04-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17791 This is very similar to https://github.com/apache/spark/pull/16804/files however that approach is like this one is slightly broken (because it does not support nested char/varchar columns), can

[GitHub] spark issue #17784: [SPARK-20492][SQL] Do not print empty parentheses for in...

2017-04-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17784 LGTM - pending jenkins --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/1 add to whitelist --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17777: [SPARK-20482][SQL] Resolving Casts is too strict on havi...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/1 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17770: [SPARK-20392][SQL][WIP] Set barrier to prevent re...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17770#discussion_r113474943 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -72,6 +72,34 @@ object CurrentOrigin

[GitHub] spark issue #17771: [SPARK-20471]Remove AggregateBenchmark testsuite warning...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17771 This is kinda trivial, but sure. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17771: [SPARK-20471]Remove AggregateBenchmark testsuite warning...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17771 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17772: [SPARK-20473] Enabling missing types in ColumnVector.Arr...

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17772 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17773: [SPARK-20474] Fixing OnHeapColumnVector reallocation

2017-04-26 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17773 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17749: [SPARK-20450] [SQL] Unexpected first-query schema infere...

2017-04-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17749 can you close? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17749: [SPARK-20450] [SQL] Unexpected first-query schema infere...

2017-04-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17749 LGTM - merging to 2.1. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17721: [SPARK-20013][SQL]merge renameTable to alterTable in Ext...

2017-04-24 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17721 @windpiger could you make sure that the freshly added events still work after this change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #17716: [SPARK-19952][SQL] Remove various analysis except...

2017-04-23 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17716#discussion_r112829206 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala --- @@ -160,7 +159,7 @@ abstract class SQLViewSuite extends

[GitHub] spark issue #17724: [SPARK-18127] Add hooks and extension points to Spark

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17724 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17720: [SPARK-20407][TESTS][BACKPORT-2.1] ParquetQuerySuite 'En...

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17720 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17716: [SPARK-19952][SQL] Remove various analysis except...

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17716#discussion_r112698843 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -636,17 +636,16 @@ class Analyzer

[GitHub] spark issue #17716: [SPARK-19952][SQL] Remove various analysis exceptions

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17716 cc @cloud-fan @ueshin @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17716: [SPARK-19952][SQL] Remove various analysis except...

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17716#discussion_r112662001 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -636,17 +636,16 @@ class Analyzer

[GitHub] spark pull request #17716: [SPARK-19952][SQL] Remove various analysis except...

2017-04-21 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17716 [SPARK-19952][SQL] Remove various analysis exceptions ## What changes were proposed in this pull request? We currently have quite a few analysis exception subclasses, the problem

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r112628499 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -547,6 +547,10 @@ valueExpression | left

[GitHub] spark pull request #17711: [SPARK-19951][SQL] Add string concatenate operato...

2017-04-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17711#discussion_r112627257 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala --- @@ -1483,4 +1483,16 @@ class SparkSqlAstBuilder(conf: SQLConf

[GitHub] spark pull request #17710: [SPARK-20420][SQL] Add events to the external cat...

2017-04-20 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17710 [SPARK-20420][SQL] Add events to the external catalog ## What changes were proposed in this pull request? It is often useful to be able to track changes to the `ExternalCatalog

[GitHub] spark issue #17705: [SPARK-20410][SQL] Make sparkConf a def in SharedSQLCont...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17705 I am merging this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17636 Merging to master/branch-2.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112504587 --- Diff: python/pyspark/sql/functions.py --- @@ -466,6 +466,14 @@ def nanvl(col1, col2): return Column(sc._jvm.functions.nanvl(_to_java_column

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112505928 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -387,6 +387,13 @@ case class

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112505585 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1007,22 +1006,38 @@ object functions { def map(cols: Column

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112505796 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala --- @@ -1007,22 +1006,38 @@ object functions { def map(cols: Column

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112511878 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala --- @@ -387,6 +387,13 @@ case class

[GitHub] spark pull request #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17708#discussion_r112509679 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/PlanParserSuite.scala --- @@ -537,5 +537,10 @@ class PlanParserSuite extends

[GitHub] spark issue #17707: [SPARK-20412] Throw ParseException from visitNonOptional...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17707 Could you add a unit test to the DDLCommandSuite? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17708: [SPARK-20413] Add new query hint NO_COLLAPSE.

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17708 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17701: [SPARK-20407][TESTS] ParquetQuerySuite 'Enabling/disabli...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17701 Merging to master/branch-2.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17707: [SPARK-20412] Throw ParseException from visitNonOptional...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17707 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #17704: [SPARK-20409][SQL] fail early if aggregate function in G...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17704 Also cherry picked this into 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17704: [SPARK-20409][SQL] fail early if aggregate function in G...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17704 LGTM - merging to master/branch-2.2 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17596: [SPARK-12837][CORE] Do not send the accumulator n...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17596#discussion_r112474482 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -308,16 +305,17 @@ private[spark] object TaskMetrics extends Logging

[GitHub] spark pull request #17596: [SPARK-12837][CORE] Do not send the accumulator n...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17596#discussion_r112474143 --- Diff: core/src/main/scala/org/apache/spark/executor/TaskMetrics.scala --- @@ -308,16 +305,17 @@ private[spark] object TaskMetrics extends Logging

[GitHub] spark issue #17701: [SPARK-20407][TESTS] ParquetQuerySuite 'Enabling/disabli...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17701 A few minor comments, otherwise LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17701: [SPARK-20407][TESTS] ParquetQuerySuite 'Enabling/...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17701#discussion_r112460255 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/test/SharedSQLContext.scala --- @@ -84,6 +85,10 @@ trait SharedSQLContext extends SQLTestUtils

[GitHub] spark pull request #17701: [SPARK-20407][TESTS] ParquetQuerySuite 'Enabling/...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17701#discussion_r112460112 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetQuerySuite.scala --- @@ -316,6 +316,39 @@ class

[GitHub] spark pull request #17705: [SPARK-20410][SQL] Make spark conf a val in Share...

2017-04-20 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17705 [SPARK-20410][SQL] Make spark conf a val in SharedSQLContext ## What changes were proposed in this pull request? It is kind of annoying that `SharedSQLContext.sparkConf` is a val when

[GitHub] spark issue #17699: [SPARK-20405][SQL] Dataset.withNewExecutionId should be ...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17699 LGTM - merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17703: [SPARK-20367] Properly unescape column names of partitio...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17703 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r112408980 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala --- @@ -41,13 +41,17 @@ class CastSuite extends

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r112408144 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTables.scala --- @@ -99,12 +99,9 @@ case class

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r112408167 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveInlineTables.scala --- @@ -99,12 +99,9 @@ case class

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r112408222 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala --- @@ -156,6 +156,7 @@ abstract class

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-20 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r112408198 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala --- @@ -98,11 +98,12 @@ case class

[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...

2017-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17666#discussion_r111952174 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -472,11 +472,11 @@ identifierComment

[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...

2017-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17666#discussion_r111951160 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -472,11 +472,11 @@ identifierComment

[GitHub] spark pull request #17666: [SPARK-20311][SQL] Support aliases for table valu...

2017-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17666#discussion_r111950902 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -472,11 +472,11 @@ identifierComment

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-18 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17641#discussion_r111935468 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala --- @@ -34,6 +33,9 @@ import

[GitHub] spark pull request #17641: [SPARK-20329][SQL] Make timezone aware expression...

2017-04-14 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17641 [SPARK-20329][SQL] Make timezone aware expression without timezone unresolved. ## What changes were proposed in this pull request? TBD ## How was this patch tested? TBD You

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-14 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17592 I cherry picked this into branch-2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17632: [SPARK-19946][TESTS][BACKPORT-2.1] DebugFilesystem.asser...

2017-04-13 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17632 Merging to branch-2.1. Thanks! Can you close the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17591 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17595: [SPARK-20283][SQL] Add preOptimizationBatches

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17595 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17591: [SPARK-20280][CORE] FileStatusCache Weigher integer over...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17591 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692833 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -94,27 +94,46 @@ private class

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692271 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileStatusCache.scala --- @@ -94,27 +94,46 @@ private class

[GitHub] spark pull request #17591: [SPARK-20280][CORE] FileStatusCache Weigher integ...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17591#discussion_r110692150 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileIndexSuite.scala --- @@ -220,6 +221,32 @@ class FileIndexSuite extends

[GitHub] spark issue #17566: [SPARK-19518][SQL] IGNORE NULLS in first / last in SQL

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17566 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17592 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17592: [SPARK-20243][TESTS] DebugFilesystem.assertNoOpenStreams...

2017-04-10 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17592 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17566: [SPARK-19518][SQL] IGNORE NULLS in first / last i...

2017-04-07 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17566#discussion_r11042 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala --- @@ -1023,6 +1024,22 @@ class AstBuilder extends

[GitHub] spark pull request #17487: [Spark-20145] Fix range case insensitive bug in S...

2017-04-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17487#discussion_r109307737 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala --- @@ -105,7 +105,7 @@ object

[GitHub] spark pull request #17487: [Spark-20145] Fix range case insensitive bug in S...

2017-04-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17487#discussion_r109307672 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/unresolved.scala --- @@ -69,9 +70,15 @@ case class UnresolvedInlineTable

[GitHub] spark pull request #17487: [Spark-20145] Fix range case insensitive bug in S...

2017-04-02 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17487#discussion_r109307609 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala --- @@ -105,7 +105,7 @@ object

[GitHub] spark pull request #17487: [Spark-20145] Fix range case insensitive bug in S...

2017-03-31 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17487#discussion_r109106180 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala --- @@ -105,7 +105,7 @@ object

[GitHub] spark pull request #17487: [Spark-20145] Fix range case insensitive bug in S...

2017-03-31 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17487#discussion_r109105632 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableValuedFunctions.scala --- @@ -105,7 +105,7 @@ object

[GitHub] spark issue #17487: [Spark-20145] Fix range case insensitive bug in SQL

2017-03-31 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17487 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request #17484: [SPARK-20160] [SQL] Move ParquetConversions and O...

2017-03-30 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17484#discussion_r109009831 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -170,6 +171,79 @@ object HiveAnalysis extends Rule[LogicalPlan

[GitHub] spark pull request #17484: [SPARK-20160] [SQL] Move ParquetConversions and O...

2017-03-30 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17484#discussion_r109009445 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala --- @@ -170,6 +171,79 @@ object HiveAnalysis extends Rule[LogicalPlan

[GitHub] spark issue #17379: [SPARK-20048][SQL] Cloning SessionState does not clone q...

2017-03-29 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17379 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108663100 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/sessionStateBuilders.scala --- @@ -134,6 +135,14 @@ abstract class

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108662869 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionState.scala --- @@ -32,15 +32,15 @@ import org.apache.spark.sql.internal

[GitHub] spark pull request #17379: [SPARK-20048][SQL] Cloning SessionState does not ...

2017-03-29 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17379#discussion_r108662296 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala --- @@ -134,6 +135,14 @@ abstract class

[GitHub] spark pull request #17457: [SPARK-20126][SQL] Remove HiveSessionState

2017-03-28 Thread hvanhovell
GitHub user hvanhovell opened a pull request: https://github.com/apache/spark/pull/17457 [SPARK-20126][SQL] Remove HiveSessionState ## What changes were proposed in this pull request? Commit https://github.com/apache/spark/commit/ea361165e1ddce4d8aa0242ae3e878d7b39f1de2 moved

[GitHub] spark issue #17457: [SPARK-20126][SQL] Remove HiveSessionState

2017-03-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17457 cc @cloud-fan @gatorsmile --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17428: [SPARK-20094][SQL] Preventing push down of IN subquery t...

2017-03-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17428 Merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #17453: [SPARK-20124][SQL] Join reorder should keep the s...

2017-03-28 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17453#discussion_r108389791 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala --- @@ -127,7 +127,8 @@ abstract class PlanTest extends

[GitHub] spark issue #17428: [SPARK-20094][SQL] Preventing push down of IN subquery t...

2017-03-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17428 LGTM - pending jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17448: [SPARK-20119][test-maven]Fix the test case fail in DataS...

2017-03-28 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17448 LGTM - merging to master. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17428: [SPARK-20094][SQL] Don't put predicate with IN su...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17428#discussion_r108220965 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala --- @@ -289,6 +289,9 @@ abstract class QueryPlan[PlanType

[GitHub] spark issue #17164: [SPARK-16844][SQL] Support codegen for sort-based aggrea...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/17164 @maropu I am not sure. I like to keep interesting PRs open. Would you be interested in doing some work on the planner? --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108131455 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlanner.scala --- @@ -42,6 +43,8 @@ class SparkPlanner( InMemoryScans

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108130945 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108127414 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -146,6 +141,11 @@ abstract class Optimizer

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108124015 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108118536 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108115663 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108115283 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108113280 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -1259,28 +1255,13 @@ class SessionCatalog

[GitHub] spark pull request #17433: [SPARK-20100][SQL] Refactor SessionState initiali...

2017-03-27 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/17433#discussion_r108112942 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SessionState.scala --- @@ -179,88 +132,295 @@ private[sql] class SessionState

<    4   5   6   7   8   9   10   11   12   13   >