[GitHub] spark pull request #22086: [SPARK-25096][SQL] Loosen nullability if the cast...

2018-08-12 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22086 [SPARK-25096][SQL] Loosen nullability if the cast is force-nullable. ## What changes were proposed in this pull request? In type coercion for complex types, if the found type is force

[GitHub] spark pull request #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to ar...

2018-08-12 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22075#discussion_r209478694 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -1852,6 +1852,11 @@ class DataFrameFunctionsSuite extends

[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...

2018-08-12 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22075 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209422954 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -231,6 +231,15 @@ object TypeCoercion

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209421292 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -231,6 +231,15 @@ object TypeCoercion

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209420696 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,186 @@ case class

[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...

2018-08-11 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22075 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-11 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209420042 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -231,6 +231,15 @@ object TypeCoercion

[GitHub] spark issue #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments...

2018-08-11 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22075 cc @cloud-fan @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22075: [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to ar...

2018-08-11 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22075 [SPARK-23908][SQL][FOLLOW-UP] Rename inputs to arguments, and add argument type check. ## What changes were proposed in this pull request? This is a follow-up pr of #21954 to address

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209308055 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -231,6 +231,15 @@ object TypeCoercion

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-10 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r209307767 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,186 @@ case class

[GitHub] spark issue #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistinct

2018-08-10 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22044 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-10 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22017 LGTM. @mgaido91 Do you have any other comments on this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistinct

2018-08-09 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22044 LGTM pending Jenkins. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistin...

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22044#discussion_r208873100 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3442,17 +3464,15 @@ case class

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208874195 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,184 @@ case class

[GitHub] spark pull request #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistin...

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22044#discussion_r208854615 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3517,56 +3510,24 @@ case class

[GitHub] spark pull request #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistin...

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22044#discussion_r208849675 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3410,6 +3410,28 @@ case class

[GitHub] spark pull request #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistin...

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22044#discussion_r208855525 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3410,6 +3410,28 @@ case class

[GitHub] spark pull request #22044: [SPARK-23912][SQL][Followup] Refactor ArrayDistin...

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22044#discussion_r208856510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3517,56 +3510,24 @@ case class

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-09 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21859 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21121: [SPARK-24042][SQL] Collection function: zip_with_index

2018-08-09 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21121 @mn-mikke I think we can close this since we've added `transform` which can take the index argument as suggested

[GitHub] spark issue #22052: [SPARK-25068][SQL] Add exists function.

2018-08-09 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22052 cc @hvanhovell @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22052: [SPARK-25068][SQL] Add exists function.

2018-08-09 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22052 [SPARK-25068][SQL] Add exists function. ## What changes were proposed in this pull request? This pr adds `exists` function which tests whether a predicate holds for one or more elements

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208811790 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -59,6 +59,12 @@ class

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208811179 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,65 @@ case class

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208811976 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -230,4 +236,56 @@ class

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208815130 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -2117,6 +2117,198 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208812207 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -230,4 +236,56 @@ class

[GitHub] spark pull request #22013: [SPARK-23939][SQL] Add transform_keys function

2018-08-09 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22013#discussion_r208811169 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,65 @@ case class

[GitHub] spark issue #22041: [SPARK-25058][SQL] Use Block.isEmpty/nonEmpty to check w...

2018-08-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22041 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208803198 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,184 @@ case class

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208803217 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,184 @@ case class

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208803239 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -225,7 +264,9 @@ trait

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208803179 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,191 @@ case class

[GitHub] spark issue #22041: [SPARK-25058][SQL] Use Block.isEmpty/nonEmpty to check w...

2018-08-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22041 cc @viirya @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark pull request #22041: [SPARK-25058][SQL] Use Block.isEmpty/nonEmpty to ...

2018-08-08 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22041 [SPARK-25058][SQL] Use Block.isEmpty/nonEmpty to check whether the code is empty or not. ## What changes were proposed in this pull request? We should use `Block.isEmpty/nonEmpty` instead

[GitHub] spark pull request #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not...

2018-08-08 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22039 [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not be exhaustive in Scala-2.12. ## What changes were proposed in this pull request? This is a follow-up pr of #22014. We still have

[GitHub] spark issue #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not be exh...

2018-08-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22039 cc @kiszk @srowen @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22035: [SPARK-23911][SQL][FOLLOW-UP] Fix examples of aggregate ...

2018-08-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22035 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-08 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21859 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21982: [SPARK-23911][SQL] Add aggregate function.

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21982#discussion_r208495280 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -270,3 +278,90 @@ case class ArrayFilter

[GitHub] spark pull request #22035: [SPARK-23911][SQL][FOLLOW-UP] Fix examples of agg...

2018-08-08 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22035 [SPARK-23911][SQL][FOLLOW-UP] Fix examples of aggregate function. ## What changes were proposed in this pull request? This pr is a follow-up pr of #21982 and fixes the examples

[GitHub] spark pull request #21982: [SPARK-23911][SQL] Add aggregate function.

2018-08-08 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21982#discussion_r208493152 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -270,3 +278,90 @@ case class ArrayFilter

[GitHub] spark issue #22033: [SPARK-23935][SQL][followup] mapEntry throws org.codehau...

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22033 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r208449418 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/AggregateInPandasExec.scala --- @@ -137,13 +135,12 @@ case class

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208444793 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,91 @@ case class

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208444629 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -2117,6 +2117,65 @@ class DataFrameFunctionsSuite extends

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208445048 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -442,3 +442,91 @@ case class

[GitHub] spark pull request #22031: [TODO][SPARK-23932][SQL] Higher order function zi...

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22031#discussion_r208445494 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala --- @@ -2117,6 +2117,65 @@ class DataFrameFunctionsSuite extends

[GitHub] spark issue #22033: [SPARK-23935][SQL][followup] mapEntry throws org.codehau...

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22033 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21986 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208282941 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/higherOrderFunctions.scala --- @@ -0,0 +1,166 @@ +/* + * Licensed

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208282971 --- Diff: sql/core/src/test/resources/sql-tests/inputs/higher-order-functions.sql --- @@ -0,0 +1,26 @@ +create or replace temporary view nested

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208282200 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed

[GitHub] spark pull request #21954: [SPARK-23908][SQL] Add transform function.

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21954#discussion_r208281900 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -0,0 +1,212 @@ +/* + * Licensed

[GitHub] spark issue #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce...

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22027 cc @viirya @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22027: [SPARK-25010][SQL][FOLLOWUP] Shuffle should also ...

2018-08-07 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/22027 [SPARK-25010][SQL][FOLLOWUP] Shuffle should also produce different values for each execution in streaming query. ## What changes were proposed in this pull request? This is a follow-up pr

[GitHub] spark issue #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21986 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208181599 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -365,3 +364,101 @@ case class

[GitHub] spark pull request #22017: [SPARK-23938][SQL] Add map_zip_with function

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/22017#discussion_r208181653 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -44,6 +44,21 @@ class

[GitHub] spark issue #22023: [SPARK-23928][TESTS][FOLLOWUP] Set seed to avoid flakine...

2018-08-07 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22023 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r208169432 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -205,29 +230,82 @@ case class

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-07 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r208170769 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -205,29 +230,82 @@ case class

[GitHub] spark issue #21937: [SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21937 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21937: [SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21937 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #21937: [SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21937#discussion_r208085448 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3767,230 +3767,160 @@ object

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207965203 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -205,29 +230,82 @@ case class

[GitHub] spark pull request #21937: [SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21937#discussion_r207962845 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3767,230 +3767,160 @@ object

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207923151 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -205,29 +230,82 @@ case class

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207921840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -123,7 +125,10 @@ trait

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21102 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21860 ok to test. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-06 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21102 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-06 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207781744 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,248 @@ object ArrayUnion

[GitHub] spark issue #21937: [WIP][SPARK-23914][SQL][follow-up] refactor ArrayUnion

2018-08-05 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21937 I'll revisit after the conflict is fixed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21937: [WIP][SPARK-23914][SQL][follow-up] refactor Array...

2018-08-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21937#discussion_r207767260 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3698,230 +3767,162 @@ object

[GitHub] spark pull request #21937: [WIP][SPARK-23914][SQL][follow-up] refactor Array...

2018-08-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21937#discussion_r207767113 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3698,230 +3767,162 @@ object

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207767226 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,242 @@ object ArrayUnion

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207766511 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,242 @@ object ArrayUnion

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-05 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207765490 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3965,6 +4034,242 @@ object ArrayUnion

[GitHub] spark pull request #21102: [SPARK-23913][SQL] Add array_intersect function

2018-08-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21102#discussion_r207723142 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -3805,3 +3801,339 @@ object ArrayUnion

[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.

2018-08-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21982 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207702606 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -123,7 +125,10 @@ trait

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207702742 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/HigherOrderFunctionsSuite.scala --- @@ -94,4 +94,53 @@ class

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207702649 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -123,7 +125,10 @@ trait

[GitHub] spark pull request #21986: [SPARK-23937][SQL] Add map_filter SQL function

2018-08-04 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21986#discussion_r207703208 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -210,3 +221,66 @@ case class

[GitHub] spark issue #21966: [SPARK-23915][SQL][followup] Add array_except function

2018-08-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21966 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-04 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21965 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.

2018-08-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21982 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21965: [SPARK-23909][SQL] Add filter function.

2018-08-03 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21965#discussion_r207485584 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -210,3 +219,54 @@ case class

[GitHub] spark pull request #21965: [SPARK-23909][SQL] Add filter function.

2018-08-03 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21965#discussion_r207485560 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -210,3 +219,54 @@ case class

[GitHub] spark issue #21982: [SPARK-23911][SQL] Add aggregate function.

2018-08-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21982 Oops, yes, I wrote a wrong jira-id. Fixed. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21982: [SPARK-23909][SQL] Add aggregate function.

2018-08-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21982 cc @hvanhovell @gatorsmile @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark pull request #21982: [SPARK-23909][SQL] Add aggregate function.

2018-08-03 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21982 [SPARK-23909][SQL] Add aggregate function. ## What changes were proposed in this pull request? This pr adds `aggregate` function which applies a binary operator to an initial state and all

[GitHub] spark issue #21965: [SPARK-23909][SQL] Add filter function.

2018-08-03 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21965 Jenkins, retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark issue #21973: [BUILD] Fix lint-python.

2018-08-02 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21973 Thanks! merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #21973: [BUILD] Fix lint-python.

2018-08-02 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21973 [BUILD] Fix lint-python. ## What changes were proposed in this pull request? This pr fixes lint-python. ## How was this patch tested? Executed lint-python manually. You

[GitHub] spark pull request #21966: [SPARK-23915][SQL][followup] Add array_except fun...

2018-08-02 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21966#discussion_r207289175 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -4077,81 +4078,84 @@ case class

<    1   2   3   4   5   6   7   8   9   10   >