[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22590 Hm, let's try to find a way to expose other parse options. I think we shouldn't allow every options available on Univocity ... --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99353/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #99353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99353/testReport)** for PR 22590 at commit [`9e3c4bd`](https://github.com/apache/spark/commit/9e3c4bda06011cf6b4d21321d8e7336495839325). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #99353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99353/testReport)** for PR 22590 at commit [`9e3c4bd`](https://github.com/apache/spark/commit/9e3c4bda06011cf6b4d21321d8e7336495839325). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5430/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/22590 @HyukjinKwon I think it is not important. but our customers need this feature. Yeah, it is better to find a way to set the arbitrary parse settings options --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22590 I wonder how important it is. I know `spark-csv` at Databricks supported different quote modes and that's gone when we ported that into Spark - the root cause was due to replacing the library from apache-common into univocity. After few years, I only saw one request about reviving the quote mode proposed here - so I suspect how important it is. Basically, @MaxGekk described my stand correctly. Can we investigate a way to set the arbitrary parse settings options? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22590 They should be documented in API doc like `DataFrameReader.scala`. For site, we should avoid doc duplication - It's a general issue to document options. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98473/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #98473 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98473/testReport)** for PR 22590 at commit [`59c1034`](https://github.com/apache/spark/commit/59c1034df0a0b1325bf169914b43ff66885350ec). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4770/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #98473 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98473/testReport)** for PR 22590 at commit [`59c1034`](https://github.com/apache/spark/commit/59c1034df0a0b1325bf169914b43ff66885350ec). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22590 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #98076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98076/testReport)** for PR 22590 at commit [`59c1034`](https://github.com/apache/spark/commit/59c1034df0a0b1325bf169914b43ff66885350ec). * This patch **fails due to an unknown error code, -9**. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98076/ Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4526/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #98076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98076/testReport)** for PR 22590 at commit [`59c1034`](https://github.com/apache/spark/commit/59c1034df0a0b1325bf169914b43ff66885350ec). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/22590 Normally, it's better to have no quotes, but in our production environment, the user requests quotes to be displayed, so we need this option. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97156/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #97156 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97156/testReport)** for PR 22590 at commit [`4e9cd6d`](https://github.com/apache/spark/commit/4e9cd6d4c46bbeec40182d6a5ac3ca8052b8830c). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22590 @10110346 Could you describe the use case when you need this, please. As @HyukjinKwon said in one of PR, `uniVocity` parser support many config options, and we cannot expose everything from underlying libraries otherwise we couple so much to third-party libs. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #97156 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97156/testReport)** for PR 22590 at commit [`4e9cd6d`](https://github.com/apache/spark/commit/4e9cd6d4c46bbeec40182d6a5ac3ca8052b8830c). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3821/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/96793/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #96793 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96793/testReport)** for PR 22590 at commit [`9c46a72`](https://github.com/apache/spark/commit/9c46a72517e5235e10ba0325b63817eefe5d71dd). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22590 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/3593/ Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark issue #22590: [SPARK-25574][SQL]Add an option `keepQuotes` for parsing...
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22590 **[Test build #96793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/96793/testReport)** for PR 22590 at commit [`9c46a72`](https://github.com/apache/spark/commit/9c46a72517e5235e10ba0325b63817eefe5d71dd). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
