[jira] [Commented] (SPARK-19834) csv escape of quote escape
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16294077#comment-16294077 ] Soonmok Kwon commented on SPARK-19834: -- Now, Spark uses univocity-parser 2.5.9. I reopen this issue and the pull request: https://github.com/apache/spark/pull/17177 > csv escape of quote escape > -- > > Key: SPARK-19834 > URL: https://issues.apache.org/jira/browse/SPARK-19834 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Soonmok Kwon >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > A DataFrame is stored in CSV format and loaded again. When there's backslash > followed by quotation mark, csv reading seems to make an error. > reference: > http://stackoverflow.com/questions/42607208/spark-csv-error-when-reading-backslash-and-quotation-mark -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19834) csv escape of quote escape
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16288025#comment-16288025 ] Soonmok Kwon commented on SPARK-19834: -- I will re-open this soon (in a week). > csv escape of quote escape > -- > > Key: SPARK-19834 > URL: https://issues.apache.org/jira/browse/SPARK-19834 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Soonmok Kwon >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > A DataFrame is stored in CSV format and loaded again. When there's backslash > followed by quotation mark, csv reading seems to make an error. > reference: > http://stackoverflow.com/questions/42607208/spark-csv-error-when-reading-backslash-and-quotation-mark -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19834) csv escape of quote escape
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247919#comment-16247919 ] Michael McAllister commented on SPARK-19834: Is there an update on when this tickdet is going to be worked on, and the uniVocity-parser get updated? > csv escape of quote escape > -- > > Key: SPARK-19834 > URL: https://issues.apache.org/jira/browse/SPARK-19834 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Soonmok Kwon >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > A DataFrame is stored in CSV format and loaded again. When there's backslash > followed by quotation mark, csv reading seems to make an error. > reference: > http://stackoverflow.com/questions/42607208/spark-csv-error-when-reading-backslash-and-quotation-mark -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19834) csv escape of quote escape
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925380#comment-15925380 ] Soonmok Kwon commented on SPARK-19834: -- To resolve this issue we need to enable uniVocity csv parser options: escapeUnquotedValues and charToEscapeQuoteEscaping. It is good to add as it is also described in univocity library's README.md but Exposing an option that has currently a little bug has a risk. Maybe it is better to close this issue for now and re-open when Spark bumps up to uniVocity version 2.4.0. > csv escape of quote escape > -- > > Key: SPARK-19834 > URL: https://issues.apache.org/jira/browse/SPARK-19834 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Soonmok Kwon >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > A DataFrame is stored in CSV format and loaded again. When there's backslash > followed by quotation mark, csv reading seems to make an error. > reference: > http://stackoverflow.com/questions/42607208/spark-csv-error-when-reading-backslash-and-quotation-mark -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-19834) csv escape of quote escape
[ https://issues.apache.org/jira/browse/SPARK-19834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15925375#comment-15925375 ] Hyukjin Kwon commented on SPARK-19834: -- Just for other guys to easily track this, I guess this is a good to do as it is also described in univocity's README.md - https://github.com/uniVocity/univocity-parsers/blob/master/README.md#escaping-quote-escape-characters However, there is a small bug in this option which was fixed in 2.4.0. So, I suggested to close this for now and bring it back when we bump up the library into 2.4.0 later. Please refer the details in the PR. Please let me know if anyone thinks differently. > csv escape of quote escape > -- > > Key: SPARK-19834 > URL: https://issues.apache.org/jira/browse/SPARK-19834 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.1.0 >Reporter: Soonmok Kwon >Priority: Minor > Original Estimate: 4h > Remaining Estimate: 4h > > A DataFrame is stored in CSV format and loaded again. When there's backslash > followed by quotation mark, csv reading seems to make an error. > reference: > http://stackoverflow.com/questions/42607208/spark-csv-error-when-reading-backslash-and-quotation-mark -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
