kevinwilfong opened a new pull request, #13125: URL: https://github.com/apache/gluten/pull/13125
## What changes are proposed in this pull request? HashAggregateExecBaseTransformer.checkType accepts ArrayType and StructType but not MapType, so an aggregate carrying a map through its result or buffer attributes fails doValidateInternal and runs on the JVM. HashAggregateExec, SortAggregateExec and ObjectHashAggregateExec are all offloaded through this one transformer, so no route avoids it, and the only trace is a GlutenFallbackReporter line reading "Found unsupported data type in aggregation expression: ...MapType...". Velox holds a map accumulator for the aggregates that can carry one -- arbitrary and the spark first / last family use NonNumericArbitrary -- so accept MapType as well. Spark has no aggregate that builds a map out of non-map input which is presumably why the gap went unnoticed: the type reaches checkType only when the data already has a map column and it is carried through a type-preserving aggregate such as first, last, any_value or max_by, or through a UDAF that produces one. The test covers first and last over a map column and fails without the change, with the query planned as a vanilla SortAggregate. ## How was this patch tested? New test in `VeloxAggregateFunctionsSuite`: `first` and `last` over a map column in a grouped query, checked with `runQueryAndCompare` so the results are compared against vanilla Spark, and `checkGlutenPlan[HashAggregateExecTransformer]` to confirm the aggregate is offloaded rather than falling back. ## Was this patch authored or co-authored using generative AI tooling? Co-authored with Claude Opus 5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
