[GitHub] spark issue #22302: [SPARK-21786][SQL][FOLLOWUP] Add compressionCodec test f...

2018-08-31 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22302 @fjh100456 Can you format the PR description cleanly to make others more understood? --- - To unsubscribe, e-mail: reviews

[GitHub] spark issue #22302: [SPARK-21786][SQL][FOLLOWUP] Add compressionCodec test f...

2018-08-31 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22302 @gatorsmile @ueshin Can you trigger this? I checked the related jira and code and, then I think these tests should be passed in master when `usingCTAS`=true

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214500639 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1464,12 +1465,14 @@ case class

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r214492145 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -48,6 +48,8 @@ class

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r214491914 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -48,6 +48,8 @@ class

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r214491144 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,36 +229,74 @@ case class RLike(left

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214490061 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,29 @@ class ExecutorSource(threadPool: ThreadPoolExecutor

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214489886 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,29 @@ class ExecutorSource(threadPool: ThreadPoolExecutor

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214489390 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,29 @@ class ExecutorSource(threadPool: ThreadPoolExecutor

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214351711 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1464,12 +1465,14 @@ case class

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214341661 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -85,12 +85,12 @@ class DataFrameFunctionsSuite extends QueryTest

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214340187 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -1730,9 +1730,8 @@ class DataFrameSuite extends QueryTest

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214338698 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1464,12 +1465,14 @@ case class

[GitHub] spark pull request #22270: [SPARK-25267][SQL][TEST] Disable ConvertToLocalRe...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22270#discussion_r214338655 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1464,12 +1465,14 @@ case class

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214311017 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/QueryPlanner.scala --- @@ -81,7 +81,7 @@ abstract class QueryPlanner

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214300480 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/QueryPlanner.scala --- @@ -81,7 +81,7 @@ abstract class QueryPlanner

[GitHub] spark pull request #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite

2018-08-31 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22300#discussion_r214266047 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/ExplainSuite.scala --- @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22048 +1 for adding `scalastyle:off nonascii` in that part. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22048 style errors, we cannot include non-ascii characters in files; ``` Scalastyle checks failed at following occurrences: [error] /home/jenkins/workspace/NewSparkPullRequestBuilder@2/core/src

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227657 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -497,6 +497,17 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #22284: [SPARK-25278][SQL] Avoid duplicated Exec nodes wh...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22284#discussion_r214227423 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/planning/QueryPlanner.scala --- @@ -81,7 +81,7 @@ abstract class QueryPlanner

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r214218549 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -469,4 +470,26 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark pull request #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in ...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22264#discussion_r214215448 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -290,6 +290,16 @@ object QueryTest { Row.fromSeq(row.toSeq.map

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 You need to handle that in both codegen(doGenCode) and interpreter(nullSafeEval) path. Also, can you add tests to check if they have the same behaviour in the limit=0/limit=-1 cases? You'd be better

[GitHub] spark issue #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in Hive te...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22264 `in 2.12` -> `in Scala 2.12` in the title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark pull request #22264: [SPARK-25256][SQL][TEST] Plan mismatch errors in ...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22264#discussion_r214044804 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/QueryTest.scala --- @@ -290,6 +290,16 @@ object QueryTest { Row.fromSeq(row.toSeq.map

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r214007983 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/RewriteSubquerySuite.scala --- @@ -52,4 +52,21 @@ class RewriteSubquerySuite

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r214005837 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/subquery.scala --- @@ -137,13 +137,21 @@ object RewritePredicateSubquery

[GitHub] spark pull request #21790: [SPARK-24544][SQL] Print actual failure cause whe...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21790#discussion_r213997672 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/NoSuchItemException.scala --- @@ -40,10 +40,11 @@ class

[GitHub] spark pull request #21790: [SPARK-24544][SQL] Print actual failure cause whe...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21790#discussion_r213997220 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/NoSuchItemException.scala --- @@ -40,10 +40,11 @@ class

[GitHub] spark pull request #21790: [SPARK-24544][SQL] Print actual failure cause whe...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21790#discussion_r213996010 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveSessionCatalog.scala --- @@ -36,6 +36,7 @@ import

[GitHub] spark issue #21790: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21790 Also, you need to add tests. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21790: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21790 @caneGuy Can you fix the title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #21790: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21790 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22141: [SPARK-25154][SQL] Support NOT IN sub-queries ins...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22141#discussion_r213988466 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/in-subquery/nested-not-in.sql --- @@ -0,0 +1,198 @@ +-- Tests NOT-IN subqueries nested

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-30 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213980110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,28 @@ case class JsonToStructs

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 Also, we might be able to extend the ANALYZE command for this purpose. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-30 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 ah, how about adding a new method in `SparkSession` e.g., `SparkSession.analyzeColumnCacheQuery`? --- - To unsubscribe, e-mail

[GitHub] spark pull request #22263: [SPARK-25269][SQL] SQL interface support specify ...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22263#discussion_r213904343 --- Diff: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 --- @@ -162,7 +162,8 @@ statement tableIdentifier

[GitHub] spark issue #22276: [SPARK-25242][SQL] make sql config setting fluent

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22276 oh, yes, this is the issue of binary compatibility... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22260: [SQL][MINOR] Fix compiling for scala 2.12

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22260 LGTM, too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213886899 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,51 @@ * limitations under

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213885777 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,28 @@ case class JsonToStructs

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213885328 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,28 @@ case class JsonToStructs

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19691 @DazhuangSu Are u there? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 Aha, I see. It is simple to match identifiers literally. So, let me wait for other developers comments. cc: @gatorsmile

[GitHub] spark issue #22204: [SPARK-25196][SQL] Analyze column statistics in cached q...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22204 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22276: [SPARK-25242][SQL] make sql config setting fluent

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22276 You need to update `project/MimaExcludes.scala` in this case. Either way, we cannot touch stable interfaces now. I think you need to wait for branch-2.4 cut

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213869119 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2794,6 +2794,27 @@ private[spark] object Utils extends Logging

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-29 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213868352 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2794,6 +2794,27 @@ private[spark] object Utils extends Logging

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-29 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22048 Also, can you add tests in `DatasetSuite`? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22232: [SPARK-25237][SQL]remove updateBytesReadWithFileS...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22232#discussion_r213543748 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileSourceSuite.scala --- @@ -0,0 +1,55 @@ +/* + * Licensed

[GitHub] spark issue #19691: [SPARK-14922][SPARK-17732][SQL]ALTER TABLE DROP PARTITIO...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/19691 @HyukjinKwon can you trigger again? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22227: [SPARK-25202] [SQL] Implements split with limit sql func...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/7 Also, you need to update split in python and R. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213540571 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,33 +229,59 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213539793 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,33 +229,59 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213539767 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,33 +229,59 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213538271 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,33 +229,59 @@ case class RLike(left

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213538010 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -229,33 +229,59 @@ case class RLike(left

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213537514 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -294,23 +294,29 @@ class Dataset[T] private[sql]( // We set a minimum

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213537463 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -294,23 +294,29 @@ class Dataset[T] private[sql]( // We set a minimum

[GitHub] spark pull request #22048: [SPARK-25108][SQL] Fix the show method to display...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22048#discussion_r213537424 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -294,23 +294,29 @@ class Dataset[T] private[sql]( // We set a minimum

[GitHub] spark issue #22260: [MINOR] Fix scala 2.12 build using collect

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22260 Can you add `[SQL][MINOR]` in the title? Also, can you narrow down the tittle cuz it is a little obscure. --- - To unsubscribe

[GitHub] spark pull request #22260: [MINOR] Fix scala 2.12 build using collect

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22260#discussion_r213536395 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ProjectionOverSchema.scala --- @@ -38,7 +38,7 @@ private[execution] case class

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 Thanks for the sum-up. I like simpler one, too. Le me just describe more to make me more understood; IIUC we have the two case: (1) fully-qualified case `/*+ MAPJOIN(testDb.t) */` and (2) non

[GitHub] spark issue #20965: [SPARK-21870][SQL] Split aggregation code into small fun...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20965 ok, I stop this work. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20965: [SPARK-21870][SQL] Split aggregation code into sm...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20965#discussion_r213274849 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -950,6 +950,15 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20965: [SPARK-21870][SQL] Split aggregation code into small fun...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20965 oh, made a mistake ... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21790: [SPARK-24544][SQL] Print actual failure cause when look ...

2018-08-28 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21790 sure, I'll check later. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-28 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r213197519 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,47 @@ object ResolveHints

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213158510 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql]( println(showString

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213158056 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -969,6 +969,22 @@ class DatasetSuite extends QueryTest

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22162 ya, sure. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213157406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql]( println(showString

[GitHub] spark pull request #22227: [SPARK-25202] [SQL] Implements split with limit s...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r213154483 --- Diff: sql/core/src/test/resources/sql-tests/inputs/string-functions.sql --- @@ -5,6 +5,10 @@ select format_string(); -- A pipe operator for string

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22162 I have much bandwidh to take it, too. Is it ok to take it over? @mgaido91 not working on this now? --- - To unsubscribe, e-mail

[GitHub] spark issue #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory operat...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21968 The change looks reasonable to me, so can you trigger tests? @gatorsmile @cloud-fan @hvanhovell --- - To unsubscribe, e-mail

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212959634 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -141,9 +141,6 @@ class

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212956571 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -48,6 +48,8 @@ class

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/6 Probably, you'd be better to file separate jira for each function. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212918991 --- Diff: sql/core/src/test/resources/sql-tests/inputs/json-functions.sql --- @@ -51,3 +51,8 @@ select from_json('[null, {"a":2}]

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212918588 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -32,29 +32,29 @@ object JacksonUtils

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212917164 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -65,6 +66,8 @@ private[sql] class JacksonGenerator

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/6 @MaxGekk btw, why did you attach this pr to the resolved jira? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22241: [SPARK-25249][CORE][TEST]add a unit test for Open...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22241#discussion_r212901257 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashMapSuite.scala --- @@ -194,4 +194,42 @@ class OpenHashMapSuite extends SparkFunSuite

[GitHub] spark pull request #22241: [SPARK-25249][TEST]add a unit test for OpenHashMa...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22241#discussion_r212894122 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashMapSuite.scala --- @@ -194,4 +194,42 @@ class OpenHashMapSuite extends SparkFunSuite

[GitHub] spark issue #22241: [SPARK-25249][TEST]add a unit test for OpenHashMap

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22241 plz add `[CORE]` in the title? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212891660 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/FailureSafeParser.scala --- @@ -15,50 +15,51 @@ * limitations under

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212887790 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,22 @@ case class JsonToStructs

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22198 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212861566 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r212857658 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -853,33 +853,47 @@ case class HashAggregateExec

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r212854595 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -853,33 +853,47 @@ case class HashAggregateExec

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21860 In the description, can you simplify the example query? e.g., ``` import org.apache.spark.sql.execution.debug._ spark.range(1).selectExpr("id AS key", "id AS value"

[GitHub] spark issue #22239: [SPARK-19355][SQL][Followup] Remove the child.outputPart...

2018-08-26 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/22239 Better to add `in global limit` in the title? Anyway, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212851693 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212850203 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212844578 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala --- @@ -47,20 +49,39 @@ object ResolveHints

[GitHub] spark pull request #22198: [SPARK-25121][SQL] Supports multi-part table name...

2018-08-26 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22198#discussion_r212844390 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -144,7 +144,7 @@ class Analyzer( lazy val

<    1   2   3   4   5   6   7   8   9   10   >