[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user viirya commented on the issue:
https://github.com/apache/spark/pull/20360
@hankim maybe like:
```
import pyspark.sql.functions as f
import uuid
df = spark.createDataFrame([(1,2), (3,4)])
f_udf = f.udf(lambda: str(uuid.uuid4()))
df2 = df.distinct().cache()
df3 = df2.withColumn("a", f_udf()).show()
```
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user hankim commented on the issue: https://github.com/apache/spark/pull/20360 is there any workaround for this? my environment hasn't upgrade to 2.3.0, but I have exact code that jira ticket has. (http://mail-archives.apache.org/mod_mbox/spark-issues/201801.mbox/%3cjira.13132665.151662246.6681.1516622520...@atlassian.jira%3E) i.e., assigning uuid after distinct() call with udf. Thank you! cc @viirya @HyukjinKwon @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20360 @HyukjinKwon Ok. I will open a backport later. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20360 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20360 @viirya, mind if I ask to open a backport to branch-2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86551/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86551 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86551/testReport)** for PR 20360 at commit [`74684a7`](https://github.com/apache/spark/commit/74684a7d10009ef970d7d674d9c695b695c5da5c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86551/testReport)** for PR 20360 at commit [`74684a7`](https://github.com/apache/spark/commit/74684a7d10009ef970d7d674d9c695b695c5da5c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/159/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20360 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20360 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86523/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86523 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86523/testReport)** for PR 20360 at commit [`5c3afbb`](https://github.com/apache/spark/commit/5c3afbbdf762411023b06348b2bfe3dbc2ff4287). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/135/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86523/testReport)** for PR 20360 at commit [`5c3afbb`](https://github.com/apache/spark/commit/5c3afbbdf762411023b06348b2bfe3dbc2ff4287). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20360 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86520/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86520/testReport)** for PR 20360 at commit [`5c3afbb`](https://github.com/apache/spark/commit/5c3afbbdf762411023b06348b2bfe3dbc2ff4287). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20360 cc @HyukjinKwon @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86520 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86520/testReport)** for PR 20360 at commit [`5c3afbb`](https://github.com/apache/spark/commit/5c3afbbdf762411023b06348b2bfe3dbc2ff4287). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/132/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86518/testReport)** for PR 20360 at commit [`b6cb621`](https://github.com/apache/spark/commit/b6cb6218e539589f37ff8648dff068bef6e810e5). * This patch **fails Python style tests**. * This patch **does not merge cleanly**. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86518/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20360 **[Test build #86518 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86518/testReport)** for PR 20360 at commit [`b6cb621`](https://github.com/apache/spark/commit/b6cb6218e539589f37ff8648dff068bef6e810e5). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20360: [SPARK-23177][SQL][PySpark] Extract zero-parameter UDFs ...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/130/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
