yyanyy opened a new pull request, #59853: URL: https://github.com/apache/spark/pull/59853
### What changes were proposed in this pull request? Resolve aggregate arguments in `ORDER BY` against the aggregate's input when a SELECT alias shadows an input column. Preserve output-alias precedence for scalar expressions, including scalar siblings of aggregate calls. Apply the same scope rule in the fixed-point and single-pass analyzers. The fixed-point analyzer retains the original SQL column names until function resolution identifies aggregate calls, then resolves their arguments before aggregate matching and type coercion. Already resolved attributes keep their expression IDs. ### Why are the changes needed? `SELECT team, min(score) AS score FROM scores GROUP BY team ORDER BY max(score)` currently binds `max(score)` to the output of `min(score)`, then pushes it into an Aggregate whose input cannot provide that alias. Analysis fails with `MISSING_ATTRIBUTES.RESOLVED_ATTRIBUTE_APPEAR_IN_OPERATION`. Aggregate arguments should instead refer to input columns. A scalar expression such as `ORDER BY abs(score)` should continue to use the SELECT alias. This addresses the same issue described in the stale, closed proposal #36265. ### Does this PR introduce _any_ user-facing change? Yes. Queries that order by an aggregate of an input column shadowed by a SELECT alias now analyze and return the groups in the requested order. ### How was this patch tested? Added a Catalyst plan regression covering both prebuilt aggregate expressions and unresolved SQL functions, plus SQL execution coverage for both analyzers. Controls cover existing aggregate reuse, qualified input, bare and scalar aliases, mixed aggregate/scalar expressions, HAVING, type-changing aliases, and nested fields. The new Catalyst regression and both analyzer SQL cases failed with `MISSING_ATTRIBUTES` before production changes and pass with the fix. The following focused suites pass (204 tests total): ```text build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.analysis.AnalysisSuite org.apache.spark.sql.catalyst.analysis.AnalysisErrorSuite' build/sbt 'sql/testOnly org.apache.spark.sql.SQLQuerySuite -- -z "ORDER BY aggregates resolve input columns"' build/sbt 'sql/testOnly org.apache.spark.sql.analysis.resolver.AggregateExpressionResolverSuite org.apache.spark.sql.analysis.resolver.AggregateResolverSuite' build/sbt 'sql/testOnly org.apache.spark.sql.SQLQueryTestSuite -- -z order-by.sql' ``` `catalyst/scalastyle`, `catalyst/Test/scalastyle`, and `sql/Test/scalastyle` also pass with zero errors or warnings. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
