[
https://issues.apache.org/jira/browse/CALCITE-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18094195#comment-18094195
]
Darpan Lunagariya (e6data computing) commented on CALCITE-7640:
---------------------------------------------------------------
My bad. Updated the description, I tried to make it as short as possible.
> Enumerable engine should execute aggregates whose implementor comes from a
> custom RexImplementorTable"
> ------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7640
> URL: https://issues.apache.org/jira/browse/CALCITE-7640
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.42.0
> Reporter: Darpan Lunagariya (e6data computing)
> Assignee: Darpan Lunagariya (e6data computing)
> Priority: Minor
> Labels: pull-request-available
>
> h2. Background
> CALCITE-7631 introduced {{RexImplementorTable}}, an interface for providing
> implementors for custom functions defined through a {{SqlOperatorTable}}. As
> part of that change, {{RexImpTable}} became the built-in implementation of
> {{RexImplementorTable}}. The scalar code-generation path was updated to use
> this composable table. {{RexToLixTranslator}}, and therefore
> {{RexExecutorImpl}}, can now use a custom {{RexImplementorTable}} chained
> ahead of the built-ins.
> The aggregate path was not updated. {{EnumerableAggregate}} and related
> aggregate code still resolve implementors directly from the {{RexImpTable}}
> singleton. Therefore, custom aggregate operators contributed through a
> {{SqlOperatorTable}} cannot be planned or executed by the enumerable engine
> unless they are known to the built-in table.
> h2. Problem
> The aggregate path consults the built-in singleton in two places:
> * *Planning time*: the {{EnumerableAggregate}} constructor checks each
> {{AggregateCall}} against {{RexImpTable.INSTANCE}} and throws
> {{InvalidRelException}} if no implementor is found.
> {{EnumerableAggregateRule}} catches this and returns {{null}}, so aggregates
> unknown to the built-ins are rejected during planning.
> * *Code-generation time*: {{AggImpState}} also resolves aggregate
> implementors from {{RexImpTable.INSTANCE}}.
> Because both paths hard-code the singleton, a chained {{RexImplementorTable}}
> containing custom aggregate implementors is ignored.
> Implementor availability is a planner-dependent decision. It depends on the
> active implementor table, which is available through the planner context, not
> through the {{EnumerableAggregate}} constructor alone.
> h2. Proposed Change
> Make aggregate implementor resolution use the composable
> {{RexImplementorTable}}, defaulting to the built-ins when no custom table is
> registered.
> * Add {{RexImplementorTables.of(RelOptCluster)}}. It returns the
> {{RexImplementorTable}} registered in the planner {{Context}}, or
> {{RexImpTable.instance()}} if none is registered.
> * Move the aggregate implementor availability check from the
> {{EnumerableAggregate}} constructor into {{EnumerableAggregateRule.convert}},
> where the planner context is available. If the active table has no
> implementor, the rule returns {{null}} and the planner may try another
> convention.
> * Keep only table-independent structural validation in the
> {{EnumerableAggregate}} constructor.
> * Thread the resolved {{RexImplementorTable}} into aggregate code generation.
> {{AggImpState}} should receive the table from its callers, including
> {{EnumerableAggregate}}, {{EnumerableSortedAggregate}}, {{EnumerableWindow}},
> and the interpreter's {{AggregateNode}}, using
> {{RexImplementorTables.of(getCluster())}}.
> With this change, planning and code generation use the same implementor
> table. If the enumerable aggregate rule accepts an aggregate, code generation
> can resolve the same implementor.
> {{RelOptCluster}} itself is not changed; it is only used to access the
> planner {{Context}}.
> h3. Backward Compatibility
> If no custom table is registered, {{RexImplementorTables.of(RelOptCluster)}}
> returns {{RexImpTable.instance()}}, so existing behavior is unchanged.
> h2. Example
> {code:sql}
> SELECT g, MY_CUSTOM_AGG(x) AS c
> FROM (VALUES ('a', 1), ('a', 2), ('b', 3)) AS t (g, x)
> GROUP BY g
> {code}
> Here, {{MY_CUSTOM_AGG}} is contributed through a {{SqlOperatorTable}}, and
> its implementation is provided by a chained {{RexImplementorTable}}. This
> pattern can support custom aggregate functions such as {{APPROX_TOP_K}},
> {{APPROX_COUNT_DISTINCT}}, and other approximate aggregates whose execution
> strategy is provided by the implementor.
> h2. Testing
> Add end-to-end enumerable tests for a custom aggregate implementor, asserting
> the actual result values.
> The tests should cover:
> * SQL execution through the {{Frameworks}} API.
> * A directly built planner using a registered custom {{RexImplementorTable}}.
> * A negative case where an unknown aggregate is rejected when no custom
> implementor table is registered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
