cloud-fan commented on code in PR #59072:
URL: https://github.com/apache/spark/pull/59072#discussion_r4118947122
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExprUtils.scala:
##########
@@ -59,12 +60,17 @@ object ExprUtils extends EvalHelper with QueryErrorsBase {
dataType.asInstanceOf[StructType]
}
- def convertToMapData(exp: Expression): Map[String, String] = exp match {
+ def convertToMapData(exp: Expression, functionName: String): Map[String,
String] = exp match {
case m: CreateMap
if AbstractMapType(
StringTypeWithCollation(supportsTrimCollation = true),
StringTypeWithCollation(supportsTrimCollation = true))
.acceptsType(m.dataType) =>
+ if (!m.foldable &&
+ !SQLConf.get.getConf(SQLConf.LEGACY_ALLOW_NON_FOLDABLE_OPTIONS)) {
Review Comment:
Agreed. The legacy switch now relaxes only foldability: before evaluation,
the normalized map must have no references, be deterministic, and contain no
Unevaluable nodes. Tests cover input columns, lambda variables, outer
references, and uuid() in both default and legacy modes.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ExprUtils.scala:
##########
@@ -59,12 +60,17 @@ object ExprUtils extends EvalHelper with QueryErrorsBase {
dataType.asInstanceOf[StructType]
}
- def convertToMapData(exp: Expression): Map[String, String] = exp match {
+ def convertToMapData(exp: Expression, functionName: String): Map[String,
String] = exp match {
case m: CreateMap
if AbstractMapType(
StringTypeWithCollation(supportsTrimCollation = true),
StringTypeWithCollation(supportsTrimCollation = true))
.acceptsType(m.dataType) =>
+ if (!m.foldable &&
Review Comment:
Updated in f75b84894ba. I used ReplaceExpressions.replace recursively rather
than prepareForEval so nested RuntimeReplaceable expressions such as
decode(encode(...)) are fully normalized. Foldability and evaluation both use
the normalized expression; nvl and nested decode tests pass by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
