[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19738 Thanks @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19738 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19738 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83857/testReport)** for PR 19738 at commit [`611cbf9`](https://github.com/apache/spark/commit/611cbf98856907e2d44fba73715e010333230ddd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83857/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19738 I saw some cases for ML input using unicode chars, but that was it. I think for the purposes here, it is maybe not necessary. I manually verified that without converting unicode Pandas column names the test fails, and so that might be good enough. What do you think @HyukjinKwon ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83857/testReport)** for PR 19738 at commit [`611cbf9`](https://github.com/apache/spark/commit/611cbf98856907e2d44fba73715e010333230ddd). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19738 > Yup, it looks so. Could we add another small test case for it as well? There is already, it's just that `str()` will convert from unicode as long as it's only ASCII characters so it passes. Do you think we should use non ASCII in tests? I'll check to see if that's done elsewhere in Spark.. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19738 Yup, it looks so. Could we add another small test case for it as well? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user BryanCutler commented on the issue:
https://github.com/apache/spark/pull/19738
Should this
https://github.com/apache/spark/pull/19738/files#diff-3b5463566251d5b09fd328738a9e9bc5R608
be this instead?
```
# If no schema supplied by user then get the names of columns only
if schema is None:
schema = [x.encode('utf-8') if not isinstance(x, str) else x for x in
data.columns]
```
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83807/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83807 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83807/testReport)** for PR 19738 at commit [`a27aaab`](https://github.com/apache/spark/commit/a27aaab14e3146872913a604510989699875f55e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83807 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83807/testReport)** for PR 19738 at commit [`a27aaab`](https://github.com/apache/spark/commit/a27aaab14e3146872913a604510989699875f55e). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83804/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19738 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83804/testReport)** for PR 19738 at commit [`1be2200`](https://github.com/apache/spark/commit/1be220036cc405eaef5acb77802a15bceb81c314). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19738 **[Test build #83804 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83804/testReport)** for PR 19738 at commit [`1be2200`](https://github.com/apache/spark/commit/1be220036cc405eaef5acb77802a15bceb81c314). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #19738: [SPARK-20791][PYTHON][FOLLOWUP] Check for unicode column...
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19738 Good catch @HyukjinKwon , this was an issue. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
