[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21581#discussion_r196016969
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3077,12 +3077,16 @@ object functions {
//
/**
- * Returns null if the array is null, true if the array contains
`value`, and false otherwise.
+ * Returns null if the array is null, true if the array contains `value`
or the content of
+ * `value` if it is of type Column, and false otherwise.
* @group collection_funcs
* @since 1.5.0
*/
def array_contains(column: Column, value: Any): Column = withExpr {
-ArrayContains(column.expr, Literal(value))
--- End diff --
Oh, I just found @maropu has pointed it out in
https://github.com/apache/spark/pull/21581#issuecomment-397928010.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21581#discussion_r196016648
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3077,12 +3077,16 @@ object functions {
//
/**
- * Returns null if the array is null, true if the array contains
`value`, and false otherwise.
+ * Returns null if the array is null, true if the array contains `value`
or the content of
+ * `value` if it is of type Column, and false otherwise.
* @group collection_funcs
* @since 1.5.0
*/
def array_contains(column: Column, value: Any): Column = withExpr {
-ArrayContains(column.expr, Literal(value))
--- End diff --
Other similar expressions are `ArrayPosition`, `ElementAt`, `ArrayRemove`.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21581#discussion_r196012357
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3077,12 +3077,16 @@ object functions {
//
/**
- * Returns null if the array is null, true if the array contains
`value`, and false otherwise.
+ * Returns null if the array is null, true if the array contains `value`
or the content of
--- End diff --
you can fix now
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
Github user chongguang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21581#discussion_r196002327
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3077,12 +3077,16 @@ object functions {
//
/**
- * Returns null if the array is null, true if the array contains
`value`, and false otherwise.
+ * Returns null if the array is null, true if the array contains `value`
or the content of
--- End diff --
Hey, thanks for the message. Do you want me to change the comment back? I
see that you have started the test, is it too late?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/21581#discussion_r195954291
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ---
@@ -3077,12 +3077,16 @@ object functions {
//
/**
- * Returns null if the array is null, true if the array contains
`value`, and false otherwise.
+ * Returns null if the array is null, true if the array contains `value`
or the content of
--- End diff --
We need to update this comment? I think `content of value` is a little
ambiguous.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #21581: [SPARK-24574][SQL] array_contains function deals ...
GitHub user chongguang opened a pull request: https://github.com/apache/spark/pull/21581 [SPARK-24574][SQL] array_contains function deals with Column type ## What changes were proposed in this pull request? For the function ```def array_contains(column: Column, value: Any): Column ``` , if we pass the `value` parameter as a Column type, it will yield a runtime exception. This PR proposes a pattern matching to detect if `value` is of type Column. If yes, it will use the .expr of the column, otherwise it will work as it used to. ## How was this patch tested? Unit test modified to cover this code change. You can merge this pull request into a Git repository by running: $ git pull https://github.com/chongguang/spark SPARK-24574 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/21581.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #21581 commit 27733f9ad56657925c176ae394114e0429aa9a0b Author: Chongguang LIU Date: 2018-06-17T18:17:15Z array_contains function deals with Column type for the second parameter. commit 28aa51554f4c730fae3c8090ac3c268e1ddfa4f8 Author: Chongguang LIU Date: 2018-06-17T19:58:57Z add unit test for Column type --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
