Github user ep1804 commented on a diff in the pull request: https://github.com/apache/spark/pull/20004#discussion_r157391233 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -482,6 +482,36 @@ class CSVSuite extends QueryTest with SharedSQLContext with SQLTestUtils { } } + test("save csv with quote escaping, using escapeQuoteEscaping option") { + withTempPath { path => + // when a string to be quoted ends with the escape character, escapeQuoteEscaping is required + val df1 = Seq( + """You are "beautiful"""", + """Yes, \in the inside\""", // ends with the escape character '\\' + """AA\\BB""", // 2 escape char + """AA""BB""", // 2 quote char + """AA\"BB\""", // 1 escape char and 1 quote char + """AA\""BB\""", // 1 escape char and 2 quote char + """AA\\"BB\""" // 2 escape char and 1 quote char + ).toDF() + + // escapeQuotes is true by default + // quote character is '\"' by default + // escape character is '\\' by default + df1.coalesce(1).write + .format("csv") + .option("escapeQuoteEscaping", "\\") --- End diff -- It fails. As follows: ``` [info] == Results == [info] !== Correct Answer - 7 == == Spark Answer - 7 == [info] !struct<_c0:string> struct<value:string> [info] [AA""BB] [AA""BB] [info] ![AA\""BB"] [AA\""BB\] [info] ![AA\"BB"] [AA\"BB\] [info] ![AA\\"BB"] [AA\\"BB\] [info] [AA\\BB] [AA\\BB] [info] ![Yes, \in the inside"] [Yes, \in the inside\] [info] [You are "beautiful"] [You are "beautiful"] (QueryTest.scala:163) ```
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org