masonh22 opened a new issue, #25660: URL: https://github.com/apache/datafusion/issues/25660
### Is your feature request related to a problem or challenge? At Coralogix, we have an optimization to share partial aggregations between queries where possible. Because of this, we will occasionally feed the state from an `AggregateUDFImpl`'s groups accumulator into it's regular, non-grouped accumulator. #22768 added an optimized groups accumulator for `approx_distinct()` that uses a different state format for groups with few distinct values. Trying to feed this state into the non-grouped accumulator gives an internal error: `"Impossibly got invalid binary array from states"`. ### Describe the solution you'd like I think we should document, and enforce with tests, that the `Accumulator` and `GroupsAccumulator` for all `AggregateUDFImpl`s are able to consume the state produced by their counterpart. As far as I can tell, there should never be such a fundamental difference in the intermediate values produced by the two accumulators that would make it infeasible to make them compatible with each other. In the one case I'm aware of where there's an incompatibility, the fix is trivial and shouldn't affect performance (PR: #25659). Adding tests to enforce this might be tricky. I think we could try to iterate over all the registered aggregate functions, try feeding inputs into their accumulators, and making sure that both variants produce the same results when consuming each other's intermediate values. Or, we can have the test force all aggregate functions to "register" valid inputs we can use to test their accumulators. ### Describe alternatives you've considered _No response_ ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
