Github user zero323 commented on the issue:
https://github.com/apache/spark/pull/16534
@rxin I am not aware of any straightforward way of separating these two,
but I focused on the docstrings anyway. The rationale is simple - I want to be
able to:
- Create packages containing UDFs.
- [Get concise syntax with
decorators](https://github.com/apache/spark/pull/16533) without need for
intermediate functions, or nesting.
- [Import UDFs without side
effects](https://github.com/apache/spark/pull/16536).
- Have docstrings and argument annotations which correspond to the function
I wrap, not a generic `UserDefinedFunctionObject` - this is what I want to
achieve here. As illustrated in the JIRA ticket what we get right now is
completely useless:
```
In [5]: ?add_one
Type: UserDefinedFunction
String form: <pyspark.sql.functions.UserDefinedFunction object at
0x7f281ed2d198>
File: ~/Spark/spark-2.0/python/pyspark/sql/functions.py
Signature: add_one(*cols)
Docstring:
User defined function in Python
.. versionadded:: 1.3
```
```
help(add_one)
Help on UserDefinedFunction in module pyspark.sql.functions object:
class UserDefinedFunction(builtins.object)
| User defined function in Python
|
| .. versionadded:: 1.3
|
| Methods defined here:
|
| __call__(self, *cols)
| Call self as a function.
|
| __del__(self)
|
| __init__(self, func, returnType, name=None)
| Initialize self. See help(type(self)) for accurate signature.
|
| ----------------------------------------------------------------------
| Data descriptors defined here:
|
| __dict__
| dictionary for instance variables (if defined)
|
| __weakref__
| list of weak references to the object (if defined)
(END)
```
REPL is definitely the main use case. Handling docs with `wraps` is much
trickier, but there are known workarounds .
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
