[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-08 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r208561224 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1437,6 +1437,16 @@ object SQLConf { .intConf

[GitHub] spark issue #21535: [SPARK-23596][SQL] Test interpreted path on encoders tes...

2018-08-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21535 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-08 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21933 LGTM cc @JoshRosen @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208437010 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r208436900 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,95 @@ +/* + * Licensed

[GitHub] spark pull request #22014: [SPARK-25036][SQL] avoid match may not be exhaust...

2018-08-07 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22014#discussion_r208436646 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/GenerateUnsafeProjection.scala --- @@ -87,7 +87,7 @@ object

[GitHub] spark issue #22033: [SPARK-23935][SQL][followup] mapEntry throw org.codehaus...

2018-08-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22033 cc @ueshin @mn-mikke --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22033: [SPARK-23935][SQL][followup] mapEntry throw org.c...

2018-08-07 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22033 [SPARK-23935][SQL][followup] mapEntry throw org.codehaus.commons.compiler.CompileException ## What changes were proposed in this pull request? This PR fixes an exception during

[GitHub] spark issue #22020: [SPARK-25041][build] upgrade genJavaDoc-plugin from 0.10...

2018-08-07 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22020 cc @ueshin @HyukjinKwon @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22020: upgrade genJavaDoc-plugin from 0.10 to 0.11

2018-08-07 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22020 upgrade genJavaDoc-plugin from 0.10 to 0.11 ## What changes were proposed in this pull request? This PR fixes a build error with sbt using Scala-2.12. Since [`genJavaDoc-plugin`] (https

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21933 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21087: [SPARK-23997][SQL] Configurable maximum number of...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21087#discussion_r208105302 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1490,6 +1495,8 @@ class SQLConf extends Serializable with Logging

[GitHub] spark pull request #21087: [SPARK-23997][SQL] Configurable maximum number of...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21087#discussion_r208103944 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/BucketedWriteSuite.scala --- @@ -48,16 +49,40 @@ abstract class BucketedWriteSuite extends

[GitHub] spark pull request #21937: [SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21937#discussion_r208067092 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3767,230 +3767,160 @@ object ArraySetLike

[GitHub] spark issue #22014: [SPARK-25036][SQL] avoid match may not be exhaustive in ...

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22014 cc @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22014: [SPARK-25036][SQL] avoid match may not be exhaust...

2018-08-06 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22014 [SPARK-25036][SQL] avoid match may not be exhaustive in Scala-2.12 ## What changes were proposed in this pull request? The PR remove the following compilation error using scala-2.12 with sbt

[GitHub] spark pull request #22012: [SPARK-25036][SQL] Should compare ExprValue.isNul...

2018-08-06 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/22012 [SPARK-25036][SQL] Should compare ExprValue.isNull with LiteralTrue/LiteralFalse ## What changes were proposed in this pull request? This PR fixes a comparison of `ExprValue.isNull

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207967923 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,248 @@ object ArrayUnion

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r207955873 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -232,6 +232,23 @@ class WholeStageCodegenSuite extends

[GitHub] spark issue #21937: [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21937 I see. Now, I am rebasing and resolving conflicts. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r207819972 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -166,7 +170,13 @@ class RangePartitioner[K : Ordering : ClassTag, V

[GitHub] spark pull request #21859: [SPARK-24900][SQL]Speed up sort when the dataset ...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21859#discussion_r207817068 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/benchmark/SmallDataSortBenchmark.scala --- @@ -0,0 +1,85 @@ +/* + * Licensed

[GitHub] spark issue #21989: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21989 Is this same as #21990? Would it be possible to close this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207814955 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark issue #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggregate.row....

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21931 Does this work when we set `30` into the parameter? I am afraid that several arrays with size `0x7fff` are allocated

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r207803961 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1437,6 +1437,15 @@ object SQLConf { .intConf

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-06 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r207803415 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1437,6 +1437,15 @@ object SQLConf { .intConf

[GitHub] spark issue #21998: [SPARK-24940][SQL] Use IntegerLiteral in ResolveCoalesce...

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21998 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19222: [SPARK-10399][SPARK-23879][CORE][SQL] Introduce multiple...

2018-08-06 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 ping @rednaxelafx --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21898 I see. got it, thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-08-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21898 Looks good to finish without failure. I am curious why 94247 is successful while 94241 was failed with the same set of test suites since they are tested using the same source revision

[GitHub] spark issue #21956: [MINOR][DOCS] Fix grammatical error in SortShuffleManage...

2018-08-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21956 Would it be possible to address the following parts for `can ...`, `could ...`, `may ...`, and `will ...`, too? `JavaSparkContextSuite.scala` ``` /** * Java apps can uses both

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207744998 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -566,6 +579,9 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207744005 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -566,6 +579,9 @@ private[spark] class TaskSchedulerImpl

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743879 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -567,4 +567,14 @@ package object config { .intConf

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743781 --- Diff: core/src/main/scala/org/apache/spark/BarrierTaskContext.scala --- @@ -80,7 +101,45 @@ class BarrierTaskContext( @Experimental @Since

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743563 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743465 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743456 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207743412 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,230 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21994: [SPARK-24529][Build][test-maven][follow-up] Add s...

2018-08-05 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21994#discussion_r207732745 --- Diff: pom.xml --- @@ -2609,6 +2609,28 @@ + +com.github.spotbugs +spotbugs

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-05 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21860 Does `SingleLevelAggregateHashMapSuite` cover this case? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 cc @ueshin @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retedt this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-04 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21994: [SPARK-24529][Build][test-maven][follow-up] Add s...

2018-08-03 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/21994 [SPARK-24529][Build][test-maven][follow-up] Add spotbugs 3.1.6 into maven build process ## What changes were proposed in this pull request? This PR tries to add spotbugs after [upgrading

[GitHub] spark issue #21970: [SPARK-24996][SQL] Use DSL in DeclarativeAggregate

2018-08-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21970 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21933#discussion_r207547545 --- Diff: core/src/main/scala/org/apache/spark/serializer/SerializerManager.scala --- @@ -70,6 +70,8 @@ private[spark] class SerializerManager

[GitHub] spark pull request #21087: [SPARK-23997][SQL] Configurable maximum number of...

2018-08-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21087#discussion_r20753 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -580,6 +580,11 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #21931: [SPARK-24978][SQL]Add spark.sql.fast.hash.aggrega...

2018-08-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21931#discussion_r207519280 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/VectorizedHashMapGenerator.scala --- @@ -83,7 +84,7 @@ class

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21965 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-08-03 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r207475539 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -232,6 +232,22 @@ class WholeStageCodegenSuite extends

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21966 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21965 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21933 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21933: [SPARK-24917][CORE] make chunk size configurable

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21933 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207350823 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request #21898: [SPARK-24817][Core] Implement BarrierTaskContext....

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21898#discussion_r207348484 --- Diff: core/src/main/scala/org/apache/spark/BarrierCoordinator.scala --- @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundation (ASF

[GitHub] spark issue #21935: [SPARK-24773] Avro: support logical timestamp type with ...

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21935 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21970: [SPARK-24996][SQL] Use DSL in DeclarativeAggregate

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21970 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21966 LGTM except some comments --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21966: [SPARK-23915][SQL][followup] Add array_except fun...

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21966#discussion_r207302241 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -4077,81 +4078,84 @@ case class

[GitHub] spark pull request #21966: [SPARK-23915][SQL][followup] Add array_except fun...

2018-08-02 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21966#discussion_r207302021 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -4077,81 +4078,84 @@ case class

[GitHub] spark issue #21948: [SPARK-24991][SQL] use InternalRow in DataSourceWriter

2018-08-02 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21948 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-31 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21937: [WIP][SPARK-23914][SQL][follow-up] refactor Array...

2018-07-31 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/21937 [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion ## What changes were proposed in this pull request? This PR refactors `ArrayUnion` based on [this suggestion](https://github.com

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-30 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r206398377 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,285 @@ object ArrayUnion

[GitHub] spark issue #21920: [SPARK-24956][BUILD][FOLLOWUP] Upgrade Maven version to ...

2018-07-30 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21920 @HyukjinKwon thank you --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r206008046 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,267 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r206002948 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,267 @@ object ArrayUnion

[GitHub] spark issue #19449: [SPARK-22219][SQL] Refactor code to get a value for "spa...

2018-07-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19449 cc @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r206000549 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,267 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205999688 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,267 @@ object ArrayUnion

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21912: [WIP][SPARK-24962][SQL] Refactor CodeGenerator.cr...

2018-07-29 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/21912 [WIP][SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeArray ## What changes were proposed in this pull request? This PR integrates handling of `UnsafeArrayData` and `GenericArrayData

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-29 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21102 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-28 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r205959224 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,234 @@ object ArrayUnion

[GitHub] spark issue #21908: [MINOR][CORE][TEST] Fix afterEach() in TastSetManagerSui...

2018-07-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21908 good catch, LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21898: [SPARK-24817][Core] Implement BarrierTaskContext.barrier...

2018-07-28 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21898 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21905: [SPARK-24956][Build][test-maven] Upgrade maven version t...

2018-07-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21905 cc @srowen @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21905: [SPARK-24956][Build][test-maven] Upgrade maven ve...

2018-07-27 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/21905 [SPARK-24956][Build][test-maven] Upgrade maven version to 3.5.4 ## What changes were proposed in this pull request? This PR updates maven version from 3.3.9 to 3.5.4. The current build

[GitHub] spark issue #21901: [SPARK-24950][SQL] DateTimeUtilsSuite daysToMillis and m...

2018-07-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21901 LGTM, too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r205930801 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -272,7 +272,7 @@ class OpenHashSet[@specialized(Long, Int) T

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r205930794 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3801,339 @@ object ArrayUnion

[GitHub] spark issue #20636: [SPARK-23415][SQL][TEST] Make behavior of BufferHolderSp...

2018-07-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/20636 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205930539 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3968,3 +3964,242 @@ object ArrayUnion

[GitHub] spark issue #21103: [SPARK-23915][SQL] Add array_except function

2018-07-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21103 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r205853523 --- Diff: core/src/main/scala/org/apache/spark/util/collection/OpenHashSet.scala --- @@ -272,7 +272,7 @@ class OpenHashSet[@specialized(Long, Int) T

[GitHub] spark pull request #21860: [SPARK-24901][SQL]Merge the codegen of RegularHas...

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21860#discussion_r205850895 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -232,6 +232,22 @@ class WholeStageCodegenSuite extends

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205685897 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,233 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205677242 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,233 @@ object ArrayUnion

[GitHub] spark issue #21802: [SPARK-23928][SQL] Add shuffle collection function.

2018-07-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21802 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205674364 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,233 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205674062 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,233 @@ object ArrayUnion

[GitHub] spark pull request #21103: [SPARK-23915][SQL] Add array_except function

2018-07-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21103#discussion_r205672980 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3799,233 @@ object ArrayUnion

<    1   2   3   4   5   6   7   8   9   10   >