[ https://issues.apache.org/jira/browse/SPARK-34003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262556#comment-17262556 ]
Apache Spark commented on SPARK-34003: -------------------------------------- User 'yaooqinn' has created a pull request for this issue: https://github.com/apache/spark/pull/31129 > Rule conflicts between PaddingAndLengthCheckForCharVarchar and > ResolveAggregateFunctions > ---------------------------------------------------------------------------------------- > > Key: SPARK-34003 > URL: https://issues.apache.org/jira/browse/SPARK-34003 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 3.1.0 > Reporter: Kent Yao > Assignee: Kent Yao > Priority: Critical > Fix For: 3.1.1 > > > ResolveAggregateFunctions is a hacky rule and it calls `executeSameContext` > to generate a `resolved agg` to determine which unresolved sort attribute > should be pushed into the agg. However, after we add the > PaddingAndLengthCheckForCharVarchar rule which will rewrite the query output, > thus, the `resolved agg` cannot match original attributes anymore. > It causes some dissociative sort attribute to be pushed in and fails the query > {code:java} > [info] Failed to analyze query: org.apache.spark.sql.AnalysisException: > expression 'testcat.t1.`v`' is neither present in the group by, nor is it an > aggregate function. Add to group by or wrap in first() (or first_value) if > you don't care which value you get.; > [info] Project [v#14, sum(i)#11L] > [info] +- Sort [aggOrder#12 ASC NULLS FIRST], true > [info] +- !Aggregate [v#14], [v#14, sum(cast(i#7 as bigint)) AS > sum(i)#11L, v#13 AS aggOrder#12] > [info] +- SubqueryAlias testcat.t1 > [info] +- Project [if ((length(v#6) <= 3)) v#6 else if > ((length(rtrim(v#6, None)) > 3)) cast(raise_error(concat(input string of > length , cast(length(v#6) as string), exceeds varchar type length > limitation: 3)) as string) else rpad(rtrim(v#6, None), 3, ) AS v#14, i#7] > [info] +- RelationV2[v#6, i#7, index#15, _partition#16] > testcat.t1 > [info] > [info] Project [v#14, sum(i)#11L] > [info] +- Sort [aggOrder#12 ASC NULLS FIRST], true > [info] +- !Aggregate [v#14], [v#14, sum(cast(i#7 as bigint)) AS > sum(i)#11L, v#13 AS aggOrder#12] > [info] +- SubqueryAlias testcat.t1 > [info] +- Project [if ((length(v#6) <= 3)) v#6 else if > ((length(rtrim(v#6, None)) > 3)) cast(raise_error(concat(input string of > length , cast(length(v#6) as string), exceeds varchar type length > limitation: 3)) as string) else rpad(rtrim(v#6, None), 3, ) AS v#14, i#7] > [info] +- RelationV2[v#6, i#7, index#15, _partition#16] > testcat.t1 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org