Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21282
cc @ueshin @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21282
[SPARK-23934][SQL] Adding map_from_entries function
## What changes were proposed in this pull request?
The PR adds the `map_from_entries` function that returns a map created from
the given
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21236#discussion_r187001475
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -118,6 +119,161 @@ case class MapValues
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186493539
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186474451
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186408884
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -405,6 +405,7 @@ object FunctionRegistry
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186410527
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186408382
--- Diff: python/pyspark/sql/functions.py ---
@@ -1798,6 +1798,22 @@ def create_map(*cols):
return Column(jc
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186410991
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -1033,6 +1033,17 @@ object functions {
@scala.annotation.varargs
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186409077
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186408332
--- Diff: python/pyspark/sql/functions.py ---
@@ -1798,6 +1798,22 @@ def create_map(*cols):
return Column(jc
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186410288
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186410897
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ComplexTypeSuite.scala
---
@@ -186,6 +186,37 @@ class ComplexTypeSuite
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186410190
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21258#discussion_r186409860
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala
---
@@ -235,6 +235,69 @@ case class CreateMap
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21244
@huaxingao Isn't the correct Jira number
[SPARK-24185](https://issues.apache.org/jira/browse/SPARK-24185
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21236
cc @ueshin @gatorsmile
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21236
[SPARK-23935][SQL] Adding map_entries function
## What changes were proposed in this pull request?
This PR adds `map_entries` function that returns an unordered array of all
entries
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21215
@maropu Really nice idea to create typed empty arrays via an `Literal`
expression! On the other hand, I feel that the end user shouldn't work with
classes from Catalyst internals if we consider
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21028#discussion_r185577745
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -378,6 +405,125 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185538873
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1229,3 +1229,98 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185540189
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1229,3 +1229,98 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185540852
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1229,3 +1229,98 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185534457
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1229,3 +1229,98 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185544657
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/DataFrameFunctionsSuite.scala ---
@@ -798,6 +798,111 @@ class DataFrameFunctionsSuite extends
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21208#discussion_r185532437
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1229,3 +1229,98 @@ case class Flatten
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21215
@lokm01 @gatorsmile @maropu @ueshin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21215
[SPARK-24148][SQL] Overloading array function to support typed empty arrays
## What changes were proposed in this pull request?
The PR proposes to overload `array` function and allow users
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21121
@rxin Oh, I see. In that case, I'm happy to close the PR. @hvanhovell Can
you confirm that the `transform` function will pass the index into lambda
functions
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21181
It makes sense. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21028#discussion_r184730604
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -19,14 +19,41 @@ package
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21028#discussion_r184700686
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -288,6 +288,114 @@ case class
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21121
@ueshin What about combining `zip_with_index` with
[`map_from_entries`](https://issues.apache.org/jira/browse/SPARK-23934
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21050#discussion_r184466157
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1059,3 +1059,78 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21050#discussion_r184471365
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -1059,3 +1059,78 @@ case class Flatten
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21073#discussion_r184452750
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CollectionExpressionsSuite.scala
---
@@ -56,6 +58,28 @@ class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21073#discussion_r184452242
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -116,6 +118,153 @@ case class MapValues
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21073#discussion_r18276
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -116,6 +118,153 @@ case class MapValues
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21073#discussion_r184451743
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -116,6 +118,153 @@ case class MapValues
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21073#discussion_r184435943
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -116,6 +118,153 @@ case class MapValues
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21028#discussion_r184379580
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -288,6 +288,114 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21121#discussion_r183991875
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -883,3 +884,157 @@ case class Concat
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21121#discussion_r183253723
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -883,3 +884,140 @@ case class Concat
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21121#discussion_r183253226
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -883,3 +884,140 @@ case class Concat
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21121#discussion_r183252854
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -883,3 +884,140 @@ case class Concat
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21121#discussion_r183220685
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -883,3 +884,139 @@ case class Concat
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21121
@gatorsmile I'm not aware of any. From user experience, I strongly feel
that such a function is missing. Escpecially, when
[transform](https://issues.apache.org/jira/browse/SPARK-23908) function
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21121
cc @gatorsmile @ueshin @kiszk
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21121
[SPARK-24042][SQL] Collection function: zip_with_index
## What changes were proposed in this pull request?
Implement function zip_with_index(array[, indexFirst]) that transforms
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21034
Thanks everybody for code reviews!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r182357725
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -353,3 +356,218 @@ case class ArrayMax
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r182047674
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -417,3 +419,179 @@ case class ArrayMax
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r181740076
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +290,231 @@ case class
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/21034
Any other comments?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r181593152
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,160 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r181456547
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,160 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r181456175
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,160 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21025#discussion_r180894697
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,70 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21024#discussion_r180892505
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +287,70 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/21034#discussion_r180686345
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -212,6 +213,96 @@ case class SortArray
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/21034
[SPARK-23926][SQL] Extending reverse function to support ArrayType arguments
## What changes were proposed in this pull request?
This PR extends reverse functions to be able to operate
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
Any idea why those tests are failing?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
Can't reproduce it locally and seems to unrelated...
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r180070286
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,165 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20938#discussion_r180065705
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,165 @@ case class
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
Any other comments?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
@maropu I've modified the solution according to your comments:
- Removed UnresolvedConcat and merged string and array concatenation into
one expression class.
- Implemented type coercion
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r178759909
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3046,6 +3036,16 @@ object functions {
ArrayContains(column.expr
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r178759108
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,152 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r178753211
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -699,3 +699,88 @@ abstract class
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20938
Rewrote test cases. @gatorsmile Please let me know if it's OK.
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
Any other comments?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/20938
[SPARK-23821][SQL] Collection function: flatten
## What changes were proposed in this pull request?
This PR adds a new collection function that transforms an array of arrays
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
@tdas Any idea how to fix failing the `RateSourceSuite.overflow`? It seems
that more PRs are facing the same problem
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
The PR [#20688](https://github.com/apache/spark/pull/20688) removes one of
the failing unit test from repo. Maybe we could solve the problem by rebasing
to current master
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
Can't reproduce any of the failing unit tests above and seem to be
unrelated...
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
retest please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r177419513
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,152 @@ case class
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
It seems that we experienced the same problem with failing
"RateSourceV2Suite.basic microbatch execution" test reported
[here](https://issues.apache.org/jira/browse/S
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r177351705
--- Diff: python/pyspark/sql/functions.py ---
@@ -1834,6 +1819,25 @@ def array_contains(col, value):
return Column(sc
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
retest please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
retest please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
Merged concat and concat_arrays functions into one via an unresolved
expression and subsequent resolution. Do you have any objections to this
approach
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r177084478
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,152 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r177059083
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
---
@@ -287,3 +289,152 @@ case class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r177053835
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -699,3 +699,88 @@ abstract class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r176847009
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala
---
@@ -699,3 +699,88 @@ abstract class
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r176841337
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -408,6 +408,7 @@ object FunctionRegistry
Github user mn-mikke commented on the issue:
https://github.com/apache/spark/pull/20858
@maropu What other libraries do you mean? I'm not aware of any library
providing this functionality on top Spark SQL.
When using Spark SQL as an ETL tool for structured and nested data
Github user mn-mikke commented on a diff in the pull request:
https://github.com/apache/spark/pull/20858#discussion_r175527998
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala
---
@@ -408,6 +408,7 @@ object FunctionRegistry
GitHub user mn-mikke opened a pull request:
https://github.com/apache/spark/pull/20858
[SPARK-23736][SQL] Implementation of the concat_arrays function
concatenating multiple array columns into one.
## What changes were proposed in this pull request?
The PR adds a logic for easy
201 - 295 of 295 matches
Mail list logo
