Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22795#discussion_r227026198
--- Diff: python/pyspark/sql/functions.py ---
@@ -3023,6 +3023,42 @@ def pandas_udf(f=None, returnType=None,
functionType=None):
conversion on returned data. The conversion is not guaranteed to
be correct and results
should be checked for accuracy by users.
"""
+
+ # The following table shows most of Pandas data and SQL type
conversions in Pandas UDFs that
+ # are not yet visible to the user. Some of behaviors are buggy and
might be changed in the near
+ # future. The table might have to be eventually documented externally.
+ # Please see SPARK-25798's PR to see the codes in order to generate
the table below.
+ #
--- End diff --
It's not supposed to be exposed and visible to users yet as I said in the
comments. Let's keep it internal for now.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
