[jira] [Resolved] (SPARK-21678) Disabling quotes while writing a dataframe
[ https://issues.apache.org/jira/browse/SPARK-21678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21678. --- Resolution: Fixed [~taransaini43] I read this and can tell you this is not what JIRA is for. I'm a committer. Please don't reopen this. http://spark.apache.org/contributing.html > Disabling quotes while writing a dataframe > -- > > Key: SPARK-21678 > URL: https://issues.apache.org/jira/browse/SPARK-21678 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Taran Saini > > Hi, > I have the my dataframe cloumn values which can contain commas, double quotes > etc. > I am transforming the dataframes in order to ensure that all the required > values are escaped. > However, on doing df.write.format("csv") > It again wraps the values in double quotes. How do I disable the same? > And even if the double quotes are there to stay why does it do the following : > {noformat} > L"\, p' Y a\, C G > {noformat} > is written as > {noformat} > "L\"\\, p' Y a\\, C G\\, H" > {noformat} > i.e double escapes the next already escaped values. > and if i myself escape like : > {noformat} > L\"\, p' Y a\, C G > {noformat} > then that is written as > {noformat} > "L\\"\\, p' Y a\\, C G\\, H" > {noformat} > How do we just disable this automatic escaping of characters? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-21678) Disabling quotes while writing a dataframe
[ https://issues.apache.org/jira/browse/SPARK-21678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21678. --- Resolution: Invalid > Disabling quotes while writing a dataframe > -- > > Key: SPARK-21678 > URL: https://issues.apache.org/jira/browse/SPARK-21678 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Taran Saini > > Hi, > I have the my dataframe cloumn values which can contain commas, double quotes > etc. > I am transforming the dataframes in order to ensure that all the required > values are escaped. > However, on doing df.write.format("csv") > It again wraps the values in double quotes. How do I disable the same? > And even if the double quotes are there to stay why does it do the following : > {noformat} > L"\, p' Y a\, C G > {noformat} > is written as > {noformat} > "L\"\\, p' Y a\\, C G\\, H" > {noformat} > i.e double escapes the next already escaped values. > and if i myself escape like : > {noformat} > L\"\, p' Y a\, C G > {noformat} > then that is written as > {noformat} > "L\\"\\, p' Y a\\, C G\\, H" > {noformat} > How do we just disable this automatic escaping of characters? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Resolved] (SPARK-21678) Disabling quotes while writing a dataframe
[ https://issues.apache.org/jira/browse/SPARK-21678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21678. --- Resolution: Invalid Questions belong on the mailing list or SO. > Disabling quotes while writing a dataframe > -- > > Key: SPARK-21678 > URL: https://issues.apache.org/jira/browse/SPARK-21678 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 2.2.0 >Reporter: Taran Saini > > Hi, > I have the my dataframe cloumn values which can contain commas, double quotes > etc. > I am transforming the dataframes in order to ensure that all the required > values are escaped. > However, on doing df.write.format("csv") > It again wraps the values in double quotes. How do I disable the same? > And even if the double quotes are there to stay why does it do the following : > L"\, p' Y a\, C G is written as "L\"\\, p' Y a\\, C G\\, H" i.e double > escapes the next already escaped values. > and if i myself escape like : > L\"\, p' Y a\, C G then that is written as "L\\"\\, p' Y a\\, C G\\, H" > How do we just disable this automatic escaping of characters? -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org