[
https://issues.apache.org/jira/browse/FLINK-40075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
dylanhz closed FLINK-40075.
---------------------------
Resolution: Won't Do
> Optimize WELFORD_M2 aggregate accumulator by using RowData instead of
> STRUCTURED POJO accumulator
> -------------------------------------------------------------------------------------------------
>
> Key: FLINK-40075
> URL: https://issues.apache.org/jira/browse/FLINK-40075
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
> Reporter: dylanhz
> Priority: Minor
>
> {{WelfordM2AggFunction}} is used for variance-related aggregates such as
> {{{}VAR_POP{}}}, {{{}VAR_SAMP{}}}, {{{}STDDEV_POP{}}}, and
> {{{}STDDEV_SAMP{}}}.
> Currently its accumulator is a structured POJO type. Using {{RowData}} as the
> accumulator could avoid unnecessary structured object conversion in generated
> aggregate code and improve performance on hot aggregate paths.
> This should be investigated as a separate optimization because changing the
> accumulator from {{STRUCTURED<WelfordM2Accumulator>}} to {{ROW<n, mean, m2>}}
> may affect compiled plan and streaming state/savepoint compatibility.
> *Validation*
> The implementation should include:
> * variance/stddev aggregate tests,
> * compiled plan restore coverage,
> * state/savepoint or serializer compatibility checks,
> * performance evidence for the reduced conversion overhead.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
