Repository: spark Updated Branches: refs/heads/master 71f38ac24 -> 2381953ab
[SPARK-25105][PYSPARK][SQL] Include PandasUDFType in the import all of pyspark.sql.functions ## What changes were proposed in this pull request? Include PandasUDFType in the import all of pyspark.sql.functions ## How was this patch tested? Run the test case from the pyspark shell from the jira [spark-25105](https://jira.apache.org/jira/browse/SPARK-25105?jql=project%20%3D%20SPARK%20AND%20component%20in%20(ML%2C%20PySpark%2C%20SQL%2C%20%22Structured%20Streaming%22)) I manually test on pyspark-shell: before: ` >>> from pyspark.sql.functions import * >>> foo = pandas_udf(lambda x: x, 'v int', PandasUDFType.GROUPED_MAP) Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'PandasUDFType' is not defined >>> ` after: ` >>> from pyspark.sql.functions import * >>> foo = pandas_udf(lambda x: x, 'v int', PandasUDFType.GROUPED_MAP) >>> ` Please review http://spark.apache.org/contributing.html before opening a pull request. Closes #22100 from kevinyu98/spark-25105. Authored-by: Kevin Yu <q...@us.ibm.com> Signed-off-by: Bryan Cutler <cutl...@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/2381953a Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/2381953a Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/2381953a Branch: refs/heads/master Commit: 2381953ab5d9e86d87a9ef118f28bc3f67d6d805 Parents: 71f38ac Author: Kevin Yu <q...@us.ibm.com> Authored: Wed Aug 22 10:16:47 2018 -0700 Committer: Bryan Cutler <cutl...@gmail.com> Committed: Wed Aug 22 10:16:47 2018 -0700 ---------------------------------------------------------------------- python/pyspark/sql/functions.py | 1 + 1 file changed, 1 insertion(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/2381953a/python/pyspark/sql/functions.py ---------------------------------------------------------------------- diff --git a/python/pyspark/sql/functions.py b/python/pyspark/sql/functions.py index f583373..d58d8d1 100644 --- a/python/pyspark/sql/functions.py +++ b/python/pyspark/sql/functions.py @@ -2931,6 +2931,7 @@ def pandas_udf(f=None, returnType=None, functionType=None): blacklist = ['map', 'since', 'ignore_unicode_prefix'] __all__ = [k for k, v in globals().items() if not k.startswith('_') and k[0].islower() and callable(v) and k not in blacklist] +__all__ += ["PandasUDFType"] __all__.sort() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
