This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 36ff716e344 [SPARK-41929][CONNECT][PYTHON] Add function `array_compact`
36ff716e344 is described below
commit 36ff716e344e0285329bdd278aad439114f461ae
Author: Ruifeng Zheng <ruife...@apache.org>
AuthorDate: Sat Jan 7 13:50:01 2023 +0900
[SPARK-41929][CONNECT][PYTHON] Add function `array_compact`
### What changes were proposed in this pull request?
`array_compact` is a new array function just added in last week, this PR
add it to connect
### Why are the changes needed?
api coverage
### Does this PR introduce _any_ user-facing change?
yes
### How was this patch tested?
added ut
Closes #39439 from zhengruifeng/connect_function_array_compact.
Authored-by: Ruifeng Zheng <ruife...@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls...@apache.org>
---
python/pyspark/sql/connect/functions.py | 7 +++++++
python/pyspark/sql/tests/connect/test_connect_function.py | 1 +
2 files changed, 8 insertions(+)
diff --git a/python/pyspark/sql/connect/functions.py
b/python/pyspark/sql/connect/functions.py
index f1505b19b80..aa760c1627b 100644
--- a/python/pyspark/sql/connect/functions.py
+++ b/python/pyspark/sql/connect/functions.py
@@ -1127,6 +1127,13 @@ def array_intersect(col1: "ColumnOrName", col2:
"ColumnOrName") -> Column:
array_intersect.__doc__ = pysparkfuncs.array_intersect.__doc__
+def array_compact(col: "ColumnOrName") -> Column:
+ return _invoke_function_over_columns("array_compact", col)
+
+
+array_compact.__doc__ = pysparkfuncs.array_compact.__doc__
+
+
def array_join(
col: "ColumnOrName", delimiter: str, null_replacement: Optional[str] = None
) -> Column:
diff --git a/python/pyspark/sql/tests/connect/test_connect_function.py
b/python/pyspark/sql/tests/connect/test_connect_function.py
index 0dda3e99fcf..bde5ca26aa8 100644
--- a/python/pyspark/sql/tests/connect/test_connect_function.py
+++ b/python/pyspark/sql/tests/connect/test_connect_function.py
@@ -957,6 +957,7 @@ class SparkConnectFunctionTests(SparkConnectFuncTestCase):
for cfunc, sfunc in [
(CF.array_distinct, SF.array_distinct),
+ (CF.array_compact, SF.array_compact),
(CF.array_max, SF.array_max),
(CF.array_min, SF.array_min),
(CF.reverse, SF.reverse),
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
