[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 I apologize I haven't had time to work on this. I can close this for now and reopen it when I have a working fix for it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20068 ping @aa8y @HyukjinKwon @MaxGekk @gengliangwang --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86739/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #86739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86739/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #86739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86739/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 ping @aa8y --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 I'll work on it in the next week or two. That would involve a PR to the Univocity CSV parser. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 Can we make the tests pass BTW, @aa8y? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85404/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85404/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85404/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85368/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85368/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 @HyukjinKwon I made code changes based on your suggestions. I also changed the tests to use the data mentioned in the ticket. However, you're right, the tests no longer pass. But that is because the Univocity `CsvParser`, when it encounters an empty string while parsing the data, replaces it with the `nullValue` we set (see [setNullValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/common/CommonSettings.html#setNullValue(java.lang.String))). And the `emptyValue` is only effective when the _empty string_ being read has quotes around it (see [setEmptyValue()](http://docs.univocity.com/parsers/2.5.9/com/univocity/parsers/csv/CsvParserSettings.html#setEmptyValue(java.lang.String))). So I believe, at this point, the issue needs to be fixed in the underlying library being used. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85368/testReport)** for PR 20068 at commit [`156d755`](https://github.com/apache/spark/commit/156d755d5a734a00c4c69dfc3565364f3843fca1). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/85353/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85353/testReport)** for PR 20068 at commit [`ebe2900`](https://github.com/apache/spark/commit/ebe2900aadd3af0114ed71506088c6a736dd5002). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20068 **[Test build #85353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/85353/testReport)** for PR 20068 at commit [`ebe2900`](https://github.com/apache/spark/commit/ebe2900aadd3af0114ed71506088c6a736dd5002). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20068 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user aa8y commented on the issue: https://github.com/apache/spark/pull/20068 @gatorsmile I've created this PR since #12904 has not been updated in a while. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #20068: [SPARK-17916][SQL] Fix empty string being parsed as null...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20068 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
