[GitHub] spark pull request #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `p...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/20900


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `p...

[HyukjinKwon]

Github user HyukjinKwon commented on a diff in the pull request:

https://github.com/apache/spark/pull/20900#discussion_r176930776
  
--- Diff: python/pyspark/sql/functions.py ---
@@ -2155,6 +2155,8 @@ def udf(f=None, returnType=StringType()):
 in boolean expressions and it ends up with being executed all 
internally. If the functions
 can fail on special rows, the workaround is to incorporate the 
condition into the functions.
 
+.. note:: The user-defined functions do not take keyword arguments.
--- End diff --

Let's add `on the calling side` (I meant not on function declaration side) 
at the end for clarification.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20900: [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `p...

[mstewart141]

GitHub user mstewart141 opened a pull request:

https://github.com/apache/spark/pull/20900

 [SPARK-23645][MINOR][DOCS][PYTHON] Add docs RE `pandas_udf` with keyword 
args

## What changes were proposed in this pull request?

Add documentation about the limitations of `pandas_udf` with keyword 
arguments and related concepts, like `functools.partial` fn objects.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/mstewart141/spark udfkw2

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20900.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20900


commit 048570f7e5f421288b7c297e4d2e3873626a6adc
Author: Michael (Stu) Stewart 
Date:   2018-03-11T20:38:29Z

[SPARK-23645][PYTHON] Allow python udfs to be called with keyword arguments

commit 9ea2595f0cecb0cd05e0e6b99baf538679332e8b
Author: Michael (Stu) Stewart 
Date:   2018-03-18T18:04:21Z

Incomplete / Show issue with partial fn in pandas_udf

commit acd1cbe53dc7d1bf83b1022a7e36652cd9530b58
Author: Michael (Stu) Stewart 
Date:   2018-03-18T18:13:53Z

Add note RE no keyword args in python UDFs

commit bc49c3cc5ae2e23da5cc7b6d7e1a779e9d012c8c
Author: Michael (Stu) Stewart 
Date:   2018-03-24T17:30:15Z

Address comments




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org