[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20348 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20348#discussion_r162861480 --- Diff: python/pyspark/sql/udf.py --- @@ -213,6 +213,10 @@ def register(self, name, f, returnType=None): `returnType` can be optionally specified when `f` is a Python function but not when `f` is a user-defined function. Please see below. +To register a non-deterministic Python function, users need to first build +a nondeterministic user-defined function for the Python function and then register it --- End diff -- `nondeterministic` is better --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20348#discussion_r162857800 --- Diff: python/pyspark/sql/udf.py --- @@ -213,6 +213,10 @@ def register(self, name, f, returnType=None): `returnType` can be optionally specified when `f` is a Python function but not when `f` is a user-defined function. Please see below. +To register a non-deterministic Python function, users need to first build --- End diff -- Shall we switch this paragraph with `` `returnType` can be optionally s ... ``? I intended to explain `returnType` with case 1. and case 2. together. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20348#discussion_r162857579 --- Diff: python/pyspark/sql/udf.py --- @@ -213,6 +213,10 @@ def register(self, name, f, returnType=None): `returnType` can be optionally specified when `f` is a Python function but not when `f` is a user-defined function. Please see below. +To register a non-deterministic Python function, users need to first build +a nondeterministic user-defined function for the Python function and then register it --- End diff -- `nondeterministic` -> `non-deterministic` or the opposite. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20348#discussion_r162856477 --- Diff: python/pyspark/sql/udf.py --- @@ -200,7 +200,7 @@ def __init__(self, sparkSession): @since("1.3.1") def register(self, name, f, returnType=None): """Registers a Python function (including lambda function) or a user-defined function --- End diff -- `Register` instead of `Registers` to be consistent with other descriptions? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #20348: [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs...
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20348 [SPARK-23122][PYSPARK][FOLLOW-UP] Update the docs for UDF Registration ## What changes were proposed in this pull request? This PR is to update the docs for UDF registration ## How was this patch tested? N/A You can merge this pull request into a Git repository by running: $ git pull https://github.com/gatorsmile/spark testUpdateDoc Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/20348.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #20348 commit 91ea1f58b5daa27232c303f1bbeca09245a1aa9f Author: gatorsmile Date: 2018-01-22T06:16:46Z update the doc. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org