cloud-fan commented on code in PR #59072:
URL: https://github.com/apache/spark/pull/59072#discussion_r4118950433
##########
sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala:
##########
@@ -439,6 +439,66 @@ class CsvFunctionsSuite extends SharedSparkSession {
checkAnswer(df1.selectExpr("to_csv(a)"), Row("1") :: Nil)
}
+ test("to_csv with non-foldable options") {
Review Comment:
Done in f75b84894ba. The test names now include SPARK-59801, and the CSV,
JSON, and XML suites cover all nine call sites. The three suites pass all 216
tests.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExprUtils.scala:
##########
@@ -59,12 +60,17 @@ object ExprUtils extends EvalHelper with QueryErrorsBase {
dataType.asInstanceOf[StructType]
}
- def convertToMapData(exp: Expression): Map[String, String] = exp match {
+ def convertToMapData(exp: Expression, functionName: String): Map[String,
String] = exp match {
case m: CreateMap
if AbstractMapType(
StringTypeWithCollation(supportsTrimCollation = true),
StringTypeWithCollation(supportsTrimCollation = true))
.acceptsType(m.dataType) =>
+ if (!m.foldable &&
+ !SQLConf.get.getConf(SQLConf.LEGACY_ALLOW_NON_FOLDABLE_OPTIONS)) {
+ throw QueryCompilationErrors.nonFoldableArgumentError(
+ functionName, "options", m.dataType)
Review Comment:
Thanks. I kept the existing NON_FOLDABLE_ARGUMENT condition as suggested,
and updated the PR description and migration guide to use it consistently.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
