[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.afterAll...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r215881992 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/WideSchemaBenchmark.scala --- @@ -54,8 +54,11 @@ class WideSchemaBenchmark

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.afterAll...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r215874969 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/ExchangeCoordinatorSuite.scala --- @@ -31,6 +31,7 @@ class ExchangeCoordinatorSuite extends

[GitHub] spark pull request #22348: [SPARK-25354][SQL] Reduce unneeded operation in n...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22348#discussion_r215860694 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java --- @@ -154,8 +159,6 @@ public void

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-09-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22347 Thank you for your update. Is it better to add test case to confirm the state of the internal structures is as you expected? @maropu

[GitHub] spark issue #22345: [SPARK-12321][SQL][FOLLOW-UP] Add tests for fromString

2018-09-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22345 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.afterAll() in a...

2018-09-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 cc @dongjoon-hyun --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22347: [SPARK-25353][SQL] Refactoring executeTake(n: Int) in Sp...

2018-09-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22347 It is fine with me for separation since this is not only for ThriftServer. It does not seem to be just refactoring. Could you please update the title of PR appropriately

[GitHub] spark issue #22337: [SPARK-25338][Test][kafka][kinesis][flume] Ensure to cal...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 Sure, I just focused on files under `external`. Let me address other files, too. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215452975 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,8 +646,17 @@ private[spark] class AppStatusListener

[GitHub] spark issue #22167: [SPARK-25170][DOC] Add list and short description of Spa...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22167 I like to add description for metrics. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315965 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315657 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215315711 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215314710 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note

[GitHub] spark pull request #22167: [SPARK-25170][DOC] Add list and short description...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22167#discussion_r215314527 --- Diff: docs/monitoring.md --- @@ -388,6 +388,163 @@ value triggering garbage collection on jobs, and `spark.ui.retainedStages` that Note

[GitHub] spark issue #22322: [SPARK-25312][Documentation, Spark Core] Add description...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22322 Thank you for adding a description for the conf. Would it be possible to add missing descriptions for other confs in `spark.network.crypto

[GitHub] spark issue #22318: [SPARK-25150][SQL] Rewrite condition when deduplicate Jo...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22318 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22338: [SPARK-25317][CORE] Avoid perf regression in Murmur3 Has...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22338 While I say these performance differences, I do not understand why these difference occurs completely. That is why I said "I have not found the root cause". Let us narrow down t

[GitHub] spark issue #22338: [SPARK-25317][CORE] Avoid perf regression in Murmur3 Has...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22338 In addition to [your commit](https://github.com/apache/spark/pull/22338/commits/91adce590461dda885d88319a700a775e63f9ce6), I applied the following change, basically use `MemoryBlock

[GitHub] spark issue #22338: [SPARK-25317][CORE] Avoid perf regression in Murmur3 Has...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22338 @mgaido91 thanks, interestingly I did experiments with similar code in my box. While I am using the linux box, I can confirm the performance improvement (or performance recover

[GitHub] spark issue #22336: [SPARK-25306][SQL][FOLLOWUP] Change `test` to `ignore` i...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22336 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22337: [SPARK-25338][Test][kafka][kinesis][flume] Ensure to cal...

2018-09-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215149786 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -646,8 +646,17 @@ private[spark] class AppStatusListener

[GitHub] spark pull request #22335: [SPARK-25091][SQL] reduce the storage memory in E...

2018-09-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22335#discussion_r215147974 --- Diff: core/src/main/scala/org/apache/spark/status/LiveEntity.scala --- @@ -33,7 +29,9 @@ import org.apache.spark.storage.RDDInfo import

[GitHub] spark pull request #22337: [SPARK-25338][Test][kafka][kinesis][flume] Ensure...

2018-09-05 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22337 [SPARK-25338][Test][kafka][kinesis][flume] Ensure to call super.afterAll() in afterAll method in test cases ## What changes were proposed in this pull request? This PR ensures to call

[GitHub] spark issue #22336: [SPARK-25306][SQL][FOLLOWUP] Change `test` to `ignore` i...

2018-09-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22336 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22306: [SPARK-25300][CORE]Unified the configuration parameter `...

2018-09-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22306 cc @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22306: [SPARK-25300][CORE]Unified the configuration parameter `...

2018-09-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22306 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22321: [DOC] Update some outdated links

2018-09-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22321 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22321: [DOC] Update the 'Specifying the Hadoop Version' link in...

2018-09-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22321 Good catch. IIUC, the following files also have the similar problem regarding `http://spark.apache.org/docs/latest/building-spark.html#building-with-buildmvn`. Would it be possible to address them

[GitHub] spark pull request #22317: [SPARK-25310][SQL] ArraysOverlap may throw a Comp...

2018-09-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22317#discussion_r214559736 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1623,12 +1623,13 @@ case class

[GitHub] spark issue #22306: [SPARK-25300][CORE]Unified the configuration parameter `...

2018-09-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22306 Good changes. Why don't we apply this changes to test cases? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22317: ArraysOverlap may throw a CompilationException

2018-09-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22317 cc @mgaido91@ueshin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22317: ArraysOverlap may throw a CompilationException

2018-09-02 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22317 ArraysOverlap may throw a CompilationException ## What changes were proposed in this pull request? ArraysOverlap may throw a CompilationException This PR fixes a problem

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-09-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21912 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22315: [SPARK-25308][SQL] ArrayContains function may return a e...

2018-09-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22315 Thank you LGTM except one comment. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #22315: [SPARK-25308][SQL] ArrayContains function may ret...

2018-09-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22315#discussion_r214542052 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -1464,17 +1464,27 @@ case class

[GitHub] spark issue #22313: [SPARK-25306][SQL] Use cache to speed up `createFilter` ...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22313 General question: Why do we use `time` instead of `entry size` to control cache? I am neutral on this decision. I would like to hear the reason of this decision

[GitHub] spark issue #22314: [SPARK-25307] ArraySort function may return an error in ...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22314 Good catch, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22219 @Dooyoung-Hwang Would it possible to add a test case to verify result with and without incrementalCollects by changing a value of `spark.sql.thriftServer.batchDeserializeLimit

[GitHub] spark issue #22221: [SPARK-25231] : Fix synchronization of executor heartbea...

2018-09-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/1 @pgandhi999 thank you for your comments. LGTM, then let me ask @zsxwing and @JoshRosen for their comments --- - To unsubscribe

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r214505265 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,29 @@ class ExecutorSource(threadPool: ThreadPoolExecutor

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22048 LGTM, then let me ask @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createU...

2018-08-31 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21912#discussion_r214392521 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -463,23 +463,35 @@ public static UnsafeArrayData

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 I see. Let me remove the change regarding `asNullable` from this PR. I will create another PR after this PR is merged

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 We need to detect the correctly written test with a wrong result. Let us think about the following `map_zip_with` without #22126. In the following example, `map_zip_with` without #22126

[GitHub] spark issue #22300: [SPARK-25296][SQL][TEST] Create ExplainSuite

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22300 SGTM. can we use this utility function in `StreamSuite`, too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20637 I believe we still need this change. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-30 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22048 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22240: [SPARK-25248] [CORE] Audit barrier Scala APIs for...

2018-08-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22240#discussion_r214229966 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -145,20 +143,77 @@ class BarrierTaskContext

[GitHub] spark issue #22292: [SPARK-25286][CORE] Removing the dangerous parmap

2018-08-30 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22292 For future, let us keep a connection with #22233 that is the original one --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createU...

2018-08-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21912#discussion_r214145047 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -385,107 +385,124 @@ case class MapEntries

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r213977445 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +35,24 @@ class

[GitHub] spark issue #22269: [SPARK-23997][SQL][Followup] Update exception message

2018-08-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22269 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22269: [SPARK-23997][SQL][Followup] Update exception mes...

2018-08-29 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22269 [SPARK-23997][SQL][Followup] Update exception message ## What changes were proposed in this pull request? This PR is an follow-up PR of #21087 based on [a discussion thread](https

[GitHub] spark pull request #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createU...

2018-08-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21912#discussion_r213707288 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -385,107 +385,120 @@ case class MapEntries

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r213689187 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +35,24 @@ class

[GitHub] spark issue #22240: [SPARK-25248] [CORE] Audit barrier Scala APIs for 2.4

2018-08-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22240 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r213645236 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -329,49 +329,52 @@ abstract class SparkPlan extends QueryPlan

[GitHub] spark issue #21087: [SPARK-23997][SQL] Configurable maximum number of bucket...

2018-08-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21087 @gatorsmile I see. I will open the PR today. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21912 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22240: [SPARK-25248] [CORE] Audit barrier Scala APIs for...

2018-08-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22240#discussion_r213377591 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -68,7 +74,7 @@ class BarrierTaskContext( * * CAUTION

[GitHub] spark issue #22246: [SPARK-25235] [SHELL] Merge the REPL code in Scala 2.11 ...

2018-08-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22246 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213371098 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213322084 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand

[GitHub] spark pull request #20965: [SPARK-21870][SQL] Split aggregation code into sm...

2018-08-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20965#discussion_r213273861 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -950,6 +950,15 @@ object SQLConf { .booleanConf

[GitHub] spark issue #22048: [SPARK-25108][SQL] Fix the show method to display the wi...

2018-08-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22048 @xuejianbest thank you for your update. Would it be possible to commit test cases, too? --- - To unsubscribe, e-mail: reviews

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213138024 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22162 Would someone please take it? I have less bandwidth next two days since I will be in a training session at my office

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r213013507 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213009058 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand

[GitHub] spark pull request #22221: [SPARK-25231] : Fix synchronization of executor h...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r213005226 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl( var

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22243 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22241 Adding tests looks good. Probably beyond the purpose of this PR, is the following difference intentionally designed? ``` // If the data type is in @specialized annotation

[GitHub] spark issue #22242: Branch 2.3

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22242 @ArunkumarRamanan would it be possible to close this? probably something is wrong. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212909243 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -164,9 +164,8 @@ private[joins] class

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r212857783 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3237,6 +3237,20 @@ class Dataset[T] private[sql]( files.toSet.toArray

[GitHub] spark issue #22219: [SPARK-25224][SQL] Improvement of Spark SQL ThriftServer...

2018-08-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22219 Would it possible to prepare test cases? IIUC, this feature can be enabled without thriftServer by writing some test code

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r212834287 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3237,6 +3237,20 @@ class Dataset[T] private[sql]( files.toSet.toArray

[GitHub] spark issue #22221: [SPARK-25231] : Fix synchronization of executor heartbea...

2018-08-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/1 We would appreciate it if you would describe some detail on `further analysis` for removing lock on the whole object. > On further analysis of the heartbeat receiver method, it turns

[GitHub] spark pull request #22221: [SPARK-25231] : Fix synchronization of executor h...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r212833953 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl( var

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212833235 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -17,11 +17,13 @@ package org.apache.spark.executor

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212832846 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -969,6 +969,22 @@ class DatasetSuite extends QueryTest

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212832850 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -969,6 +969,22 @@ class DatasetSuite extends QueryTest

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212832795 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala --- @@ -969,6 +969,22 @@ class DatasetSuite extends QueryTest

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212832804 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql]( println(showString

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r212832734 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql]( println(showString

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212832157 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r212831774 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-25 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212805339 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21931 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21087: [SPARK-23997][SQL] Configurable maximum number of bucket...

2018-08-25 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21087 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21087: [SPARK-23997][SQL] Configurable maximum number of bucket...

2018-08-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21087 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21087: [SPARK-23997][SQL] Configurable maximum number of bucket...

2018-08-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21087 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-24 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21912 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r212706680 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -348,30 +350,30 @@ abstract class SparkPlan extends QueryPlan

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212683385 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r212664629 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperation.scala --- @@ -289,6 +289,19 @@ private

[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-24 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r212664660 --- Diff: sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkExecuteStatementOperation.scala --- @@ -289,6 +289,19 @@ private

<    1   2   3   4   5   6   7   8   9   10   >