Github user maropu commented on the issue: https://github.com/apache/spark/pull/19977 I found the optimizer rule can't combine nested concat like; ``` scala>: psate val df = sql(""" SELECT ((col1 || col2) || (col3 || col4)) col FROM ( SELECT encode(string(id), 'utf-8') col1, encode(string(id + 1), 'utf-8') col2, string(id + 2) col3, string(id + 3) col4 FROM range(10) ) """) scala> df.explain(true) == Parsed Logical Plan == 'Project [concat(concat('col1, 'col2), concat('col3, 'col4)) AS col#4]+- 'SubqueryAlias __auto_generated_subquery_name +- 'Project ['encode('string('id), utf-8) AS col1#0, 'encode('string(('id + 1)), utf-8) AS col2#1, 'string(('id + 2)) AS col3#2, 'string(('id + 3)) AS col4#3 ] +- 'UnresolvedTableValuedFunction range, [10] == Analyzed Logical Plan == col: string Project [concat(cast(concat(col1#0, col2#1) as string), concat(col3#2, col4#3)) AS col#4] +- SubqueryAlias __auto_generated_subquery_name +- Project [encode(cast(id#9L as string), utf-8) AS col1#0, encode(cast((id#9L + cast(1 as bigint)) as string), utf-8) AS col2#1, cast((id#9L + cast(2 as bigint)) as string) AS col3#2, cast((id#9L + cast(3 as bigint)) as string) AS col4#3] +- Range (0, 10, step=1, splits=None) == Optimized Logical Plan == Project [concat(cast(concat(encode(cast(id#9L as string), utf-8), encode(cast((id#9L + 1) as string), utf-8)) as string), cast((id#9L + 2) as string), cast((id#9L + 3) as string)) AS col#4] +- Range (0, 10, step=1, splits=None) == Physical Plan == *Project [concat(cast(concat(encode(cast(id#9L as string), utf-8), encode(cast((id#9L + 1) as string), utf-8)) as string), cast((id#9L + 2) as string), cast((id#9L + 3) as string)) AS col#4] +- *Range (0, 10, step=1, splits=4) ``` We need to support the optimization in this case, too?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org