Jackie-Jiang opened a new pull request, #20098: URL: https://github.com/apache/pinot/pull/20098
## Summary Follow-up to apache/pinot#20074, which made each final result the stored value of its final result column type. This PR implements `mergeFinalResult` for every aggregation function whose final results can be merged, and fixes the merges that were wrong. Final results are merged when servers or multi-stage leaf workers return them (`serverReturnFinalResult`, `serverReturnFinalResultKeyUnpartitioned`, the `is_leaf_return_final_result` hint), which is only done across partitions of the aggregated values. New merges: - `HISTOGRAM` and `SUMARRAYLONG` / `SUMARRAYDOUBLE` add the arrays element-wise. - `ARRAYAGG` (every type, distinct included) concatenates the arrays, and `LISTAGG` (distinct included) joins the strings with the separator. The distinct variants rely on the partitions sharing no value. - `ANYVALUE` keeps the first value, `IDSET` unions the id sets and `STUNION` unions the geometries. - The sketch functions that return serialized sketches (`DISTINCTCOUNTRAWHLL` / `RAWHLLPLUS` / `RAWINTEGERSUMTUPLESKETCH`, `PERCENTILERAWEST` / `RAWTDIGEST` / `RAWKLL` and their MV variants, `FREQUENTLONGSSKETCH` / `FREQUENTSTRINGSSKETCH`) deserialize the two sketches, merge them, and serialize the result the same way. Fixed merges: - `DISTINCTCOUNTRAWCPCSKETCH`, `DISTINCTCOUNTRAWULL` and `DISTINCTCOUNTRAWTHETASKETCH` inherited the `LONG` merge of their non-raw parents and failed with `ClassCastException`. They now union the sketches. - `SUMPRECISION` rounds the merged sum to the precision and scale of the function, as `extractFinalResult` does. - `AVGVALUEINTEGERSUMTUPLESKETCH` overrides the merge it would otherwise inherit from the raw tuple sketch function, and keeps rejecting it. Also fixed: `DistinctCountRawULLAggregationFunction.getType()` returned `DISTINCTCOUNTULL`, so its result column was named `distinctcountull(col)`, the same as the non-raw function. It now returns `DISTINCTCOUNTRAWULL`. The broker regenerates column names from its own functions, and the non-scan and star-tree paths already treat both types the same way, so the change is safe in mixed-version clusters. `AggregationFunction#mergeFinalResult` now documents that final results are merged in the form they are transported in: a `STRING` final result is a string even where `extractFinalResult` returns an object such as `SerializedHLL`. The raw sketch functions therefore merge string forms into a string form, and their final result type parameter becomes `Comparable<?>`. Unchanged: the single-stage engine still cannot merge `SUMPRECISION` final results, because it transports them as `STRING` while the function merges `BigDecimal` (the TODO on `AggregationFunction#extractFinalResult`). Functions whose final results cannot be merged (`AVG`, `VARIANCE`, `MODE`, the non-raw percentiles, `FIRSTWITHTIME`, ...) keep throwing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
