[GitHub] spark issue #22657: [SPARK-25670][TEST] Reduce number of tested timezones in...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22657 Do we need to apply the same reduction to `CastSuite`? I believe that `DateTimeUtilsSuite` should check all of the time zones, but may skip some values between`-2` to `2

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-10-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22347 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r22445 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224333028 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224332828 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r224332984 --- Diff: docs/building-spark.md --- @@ -272,3 +272,31 @@ For SBT, specify a complete scala version using (e.g. 2.12.6): ./build/sbt -Dscala.version

[GitHub] spark pull request #22678: [SPARK-25685][BUILD] Allow running tests in Jenki...

2018-10-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22678#discussion_r223917766 --- Diff: dev/run-tests-jenkins.py --- @@ -39,7 +39,8 @@ def print_err(msg): def post_message_to_github(msg, ghprb_pull_id): print("Attem

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223783598 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22630 Thanks! merging to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

spark git commit: [SPARK-25497][SQL] Limit operation within whole stage codegen should not consume all the inputs

2018-10-09 Thread kiszk
Repository: spark Updated Branches: refs/heads/master 46fe40838 -> e3133f4ab [SPARK-25497][SQL] Limit operation within whole stage codegen should not consume all the inputs ## What changes were proposed in this pull request? This PR is inspired by https://github.com/apache/spark/pull/22524,

[GitHub] spark issue #22630: [SPARK-25497][SQL] Limit operation within whole stage co...

2018-10-09 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22630 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223478828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,68 @@ case class RangeExec(range

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -345,6 +345,61 @@ trait CodegenSupport extends SparkPlan

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223474853 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -452,46 +452,73 @@ case class RangeExec(range

[GitHub] spark pull request #22630: [SPARK-25497][SQL] Limit operation within whole s...

2018-10-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22630#discussion_r223468018 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/metric/SQLMetricsSuite.scala --- @@ -518,56 +521,81 @@ class SQLMetricsSuite extends

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223214130 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223169695 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r223169637 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CodeGenerationSuite.scala --- @@ -113,7 +113,7 @@ class CodeGenerationSuite

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r222741323 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -221,6 +227,12 @@ trait

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-10-04 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r222740882 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -69,11 +69,17 @@ trait

[GitHub] spark issue #22375: [SPARK-25388][Test][SQL] Detect incorrect nullable of Da...

2018-10-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22375 ping @cloud-fan @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

2018-10-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22602 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22602: [SPARK-25538][SQL] Zero-out all bytes when writin...

2018-10-01 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22602#discussion_r221792922 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/codegen/UnsafeRowWriter.java --- @@ -185,13 +185,13 @@ public void write(int

[GitHub] spark issue #22602: [SPARK-25538][SQL] Zero-out all bytes when writing decim...

2018-10-01 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22602 Thank you. The first option looks good. Let me think about a good UT, too. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22577: [CORE][MINOR] Fix obvious error and compiling for Scala ...

2018-09-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22577 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22579: [SPARK-25429][SQL] Use Set instead of Array to im...

2018-09-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22579#discussion_r221202958 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListener.scala --- @@ -83,7 +83,7 @@ class SQLAppStatusListener

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r221194592 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark pull request #22569: [SPARK-25542][Core][Test] Move flaky test in Open...

2018-09-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22569#discussion_r221147152 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala --- @@ -255,4 +255,16 @@ class OpenHashSetSuite extends SparkFunSuite

[GitHub] spark pull request #22569: [SPARK-25542][Core][Test] Move flaky test in Open...

2018-09-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22569#discussion_r221130737 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala --- @@ -255,4 +255,16 @@ class OpenHashSetSuite extends SparkFunSuite

[GitHub] spark issue #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenHashMapS...

2018-09-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22569 LGTM except one minor comment --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22569: [SPARK-25542][SQL][Test] Move flaky test in OpenH...

2018-09-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22569#discussion_r220954056 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashSetSuite.scala --- @@ -255,4 +255,16 @@ class OpenHashSetSuite extends SparkFunSuite

[GitHub] spark issue #22361: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-...

2018-09-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22361 It looks very intersting talk. Since I cannot join the SAIS unfortunally, I will watch this in live stream. --- - To unsubscribe

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r219448432 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r219432959 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark issue #22497: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmar...

2018-09-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22497 I see. I will wait in other PRs. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r219397495 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark issue #22497: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmar...

2018-09-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22497 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

spark git commit: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmark

2018-09-21 Thread kiszk
Repository: spark Updated Branches: refs/heads/master 596af211a -> 1f4ca6f5c [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmark ## What changes were proposed in this pull request? Refactor PrimitiveArrayBenchmark to use main method and print the output as a separate file. Run blow

[GitHub] spark issue #22497: [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmar...

2018-09-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22497 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

spark-website git commit: Add Kazuaki Ishizaki to the list of committers

2018-09-18 Thread kiszk
Repository: spark-website Updated Branches: refs/heads/asf-site 370d99580 -> 9b21d71d2 Add Kazuaki Ishizaki to the list of committers Author: Kazuaki Ishizaki Closes #147 from kiszk/addCommitter. Project: http://git-wip-us.apache.org/repos/asf/spark-website/repo Commit: http://git-

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r218635691 --- Diff: common/network-common/src/test/java/org/apache/spark/network/ChunkFetchIntegrationSuite.java --- @@ -143,37 +143,38 @@ public void releaseBuffers

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r218629945 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r218121426 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/CLIService.java --- @@ -146,16 +146,11 @@ public UserGroupInformation getHttpUGI

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r218117766 --- Diff: common/network-common/src/test/java/org/apache/spark/network/ChunkFetchIntegrationSuite.java --- @@ -143,37 +143,38 @@ public void releaseBuffers

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r218106457 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark pull request #22165: [SPARK-25017][Core] Add test suite for BarrierCoo...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22165#discussion_r218093991 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -84,20 +88,20 @@ private[spark] class BarrierCoordinator

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r218090048 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +74,60 @@ case class CreateArray

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r218085246 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r218029860 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r217997181 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-17 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22439#discussion_r217996706 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -75,87 +75,60 @@ case class CreateArray

[GitHub] spark pull request #22439: [SPARK-25444][SQL] Refactor GenArrayData.genCodeT...

2018-09-16 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22439 [SPARK-25444][SQL] Refactor GenArrayData.genCodeToCreateArrayData method ## What changes were proposed in this pull request? This PR makes `GenArrayData.genCodeToCreateArrayData` method

[GitHub] spark pull request #22375: [SPARK-25388][Test][SQL] Detect incorrect nullabl...

2018-09-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r217454434 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelperSuite.scala --- @@ -35,6 +36,13 @@ class

[GitHub] spark issue #22397: [SPARK-25170][DOC] Add list and short description of Spa...

2018-09-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22397 Yes, time unit looks confusing, but not a problem. In general, due to APIs, elapsed time is `ms`, and CPU time is `ns

[GitHub] spark issue #22375: [SPARK-25388][Test][SQL] Detect incorrect nullable of Da...

2018-09-13 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22375 cc @cloud-fan @mgaido91 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-13 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r217368066 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216968925 --- Diff: sql/catalyst/src/test/java/org/apache/spark/sql/catalyst/expressions/RowBasedKeyValueBatchSuite.java --- @@ -321,20 +302,18 @@ public void

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216968827 --- Diff: sql/catalyst/src/test/java/org/apache/spark/sql/catalyst/expressions/RowBasedKeyValueBatchSuite.java --- @@ -293,18 +276,16 @@ public void

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216968497 --- Diff: sql/catalyst/src/test/java/org/apache/spark/sql/catalyst/expressions/RowBasedKeyValueBatchSuite.java --- @@ -356,49 +335,45 @@ public void

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216968270 --- Diff: sql/hive-thriftserver/src/main/java/org/apache/hive/service/cli/CLIService.java --- @@ -146,16 +146,11 @@ public UserGroupInformation getHttpUGI

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216962458 --- Diff: core/src/test/java/test/org/apache/spark/JavaAPISuite.java --- @@ -997,10 +997,10 @@ public void binaryFiles() throws Exception

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216962519 --- Diff: core/src/test/java/test/org/apache/spark/JavaAPISuite.java --- @@ -1018,10 +1018,10 @@ public void binaryFilesCaching() throws Exception

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216962247 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -181,42 +181,43 @@ private void writeSortedFile(boolean

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216960624 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -98,19 +98,19 @@ public void

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216960499 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleBlockResolverSuite.java --- @@ -98,19 +98,19 @@ public void

[GitHub] spark issue #22397: [SPARK-25170][DOC] Add list and short description of Spa...

2018-09-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22397 LGTM cc @jiangxb1987 @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22388: Revert [SPARK-24882][SQL] improve data source v2 API fro...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22388 Incorrect import still may exist there. ``` [error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216868343 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-11 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216868299 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,8 @@ trait

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-11 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21860 LGTM cc @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nullable ...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22375 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #22347: [SPARK-25353][SQL] executeTake in SparkPlan is modified ...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22347 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-10 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216418822 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamingAggregationSuite.scala --- @@ -50,8 +50,11 @@ class StreamingAggregationSuite

[GitHub] spark pull request #22348: [SPARK-25354][SQL] Reduce unneeded operation in n...

2018-09-10 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22348#discussion_r216405506 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java --- @@ -154,8 +159,6 @@ public void

[GitHub] spark issue #22366: [SPARK-25384][SQL] Removing of spark.sql.fromJsonForceNu...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22366 I think that it is good to describe the flag is removed. This is because the default behavior is not changed and users can know the flag is not effective in the next release. cc @HyukjinKwon

[GitHub] spark issue #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nullable ...

2018-09-10 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22375 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test][SQL] Detect incorrect nu...

2018-09-10 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22375#discussion_r216206769 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,9 +223,9 @@ trait

[GitHub] spark pull request #22375: [WIP][SPARK-25388][Test] Detect incorrect nullabl...

2018-09-09 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22375 [WIP][SPARK-25388][Test] Detect incorrect nullable of DataType in the result ## What changes were proposed in this pull request? This PR can correctly cause assertion failure when incorrect

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-09-09 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r216156188 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkHadoopUtilSuite.scala --- @@ -77,6 +80,51 @@ class SparkHadoopUtilSuite extends SparkFunSuite

[GitHub] spark issue #22366: [SPARK-25384][SQL] Removing of spark.sql.fromJsonForceNu...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22366 Is it better to add a description to `docs/sql-programming-guide.md`? --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22337: [SPARK-25338][Test] Ensure to call super.beforeAll() and...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22337 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21791: [SPARK-24925][SQL] input bytesRead metrics fluctuate fro...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21791 I see. thank you for sharing the dependency and status. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22335: [SPARK-25091][Core] reduce the storage memory in Executo...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22335 @cfangplus just for your information, #22341 is modifying the similar parts in `AppStatusListener.scala`. --- - To unsubscribe, e

[GitHub] spark issue #21791: [SPARK-24925][SQL] input bytesRead metrics fluctuate fro...

2018-09-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21791 gentle ping @yucai --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216120896 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDDSuite.scala --- @@ -49,8 +49,11 @@ class StateStoreRDDSuite

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216120240 --- Diff: external/kafka-0-10/src/test/scala/org/apache/spark/streaming/kafka010/KafkaRDDSuite.scala --- @@ -44,20 +44,25 @@ class KafkaRDDSuite extends

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216120228 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaRelationSuite.scala --- @@ -51,8 +51,8 @@ class KafkaRelationSuite

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.beforeAl...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r216119902 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDDSuite.scala --- @@ -49,8 +49,11 @@ class StateStoreRDDSuite

[GitHub] spark issue #22361: Revert [SPARK-10399] [SPARK-23879] [SPARK-23762] [SPARK-...

2018-09-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22361 @gatorsmile Overall, I agree with revert since performance degradation is confirmed. When I run the TPC-DS in #19222, I have not seen such a performance regression as [here](https

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r215907441 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -853,33 +861,50 @@ case class HashAggregateExec

[GitHub] spark issue #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory operat...

2018-09-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21968 LGTM cc @cloud-fan @hvanhovell @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r215905122 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -48,6 +48,12 @@ class

[GitHub] spark pull request #21618: [SPARK-20408][SQL] Get the glob path in parallel ...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21618#discussion_r215903958 --- Diff: core/src/test/scala/org/apache/spark/deploy/SparkHadoopUtilSuite.scala --- @@ -77,6 +80,51 @@ class SparkHadoopUtilSuite extends SparkFunSuite

[GitHub] spark pull request #21221: [SPARK-23429][CORE] Add executor memory metrics t...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21221#discussion_r215899049 --- Diff: core/src/main/java/org/apache/spark/SparkFirehoseListener.java --- @@ -103,6 +103,12 @@ public final void onExecutorMetricsUpdate( onEvent

[GitHub] spark pull request #22355: [SPARK-25358][SQL] MutableProjection supports fal...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22355#discussion_r215897598 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/InterpretedMutableProjection.scala --- @@ -0,0 +1,83

[GitHub] spark issue #22355: [SPARK-25358][SQL] MutableProjection supports fallback t...

2018-09-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22355 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22337: [SPARK-25338][Test] Ensure to call super.afterAll...

2018-09-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22337#discussion_r215894224 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDDSuite.scala --- @@ -49,8 +49,11 @@ class StateStoreRDDSuite

<    1   2   3   4   5   6   7   8   9   10   >