Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20137#discussion_r159578328
--- Diff: python/pyspark/sql/catalog.py ---
@@ -227,15 +227,15 @@ def dropGlobalTempView(self, viewName):
@ignore_unicode_prefix
@since(2.0)
def registerFunction(self, name, f, returnType=StringType()):
- """Registers a python function (including lambda function) as a UDF
+ """Registers a Python function (including lambda function) or a
wrapped/native UDF
--- End diff --
It indeed added some complexity. However, I believe nothing is blocked by
#16534 now if I understand correctly.
The changes #16534 is quite nice because IMHO Python guys probably use
`help()` and `dir()` more frequently then reading the API doc in the website.
For the set of UDFs are provided as a library, I think that's quite worth to
keep.
How about leaving this wrapper logic as is for now and then we bring this
discussion back when actually something is blocked (or being too complicated)
by this?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
