[
https://issues.apache.org/jira/browse/CALCITE-7640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Darpan Lunagariya (e6data computing) updated CALCITE-7640:
----------------------------------------------------------
Description:
h2. Background
CALCITE-7631 introduced {{RexImplementorTable}}, an interface for providing
implementors for custom functions defined through a {{SqlOperatorTable}}. As
part of that change, {{RexImpTable}} became the built-in implementation of
{{RexImplementorTable}}. The scalar code-generation path was updated to use
this composable table. {{RexToLixTranslator}}, and therefore
{{RexExecutorImpl}}, can now use a custom {{RexImplementorTable}} chained ahead
of the built-ins.
The aggregate path was not updated. {{EnumerableAggregate}} and related
aggregate code still resolve implementors directly from the {{RexImpTable}}
singleton. Therefore, custom aggregate operators contributed through a
{{SqlOperatorTable}} cannot be planned or executed by the enumerable engine
unless they are known to the built-in table.
h2. Problem
The aggregate path consults the built-in singleton in two places:
* *Planning time*: the {{EnumerableAggregate}} constructor checks each
{{AggregateCall}} against {{RexImpTable.INSTANCE}} and throws
{{InvalidRelException}} if no implementor is found. {{EnumerableAggregateRule}}
catches this and returns {{null}}, so aggregates unknown to the built-ins are
rejected during planning.
* *Code-generation time*: {{AggImpState}} also resolves aggregate implementors
from {{RexImpTable.INSTANCE}}.
Because both paths hard-code the singleton, a chained {{RexImplementorTable}}
containing custom aggregate implementors is ignored.
Implementor availability is a planner-dependent decision. It depends on the
active implementor table, which is available through the planner context, not
through the {{EnumerableAggregate}} constructor alone.
h2. Proposed Change
Make aggregate implementor resolution use the composable
{{RexImplementorTable}}, defaulting to the built-ins when no custom table is
registered.
* Add {{RexImplementorTables.of(RelOptCluster)}}. It returns the
{{RexImplementorTable}} registered in the planner {{Context}}, or
{{RexImpTable.instance()}} if none is registered.
* Move the aggregate implementor availability check from the
{{EnumerableAggregate}} constructor into {{EnumerableAggregateRule.convert}},
where the planner context is available. If the active table has no implementor,
the rule returns {{null}} and the planner may try another convention.
* Keep only table-independent structural validation in the
{{EnumerableAggregate}} constructor.
* Thread the resolved {{RexImplementorTable}} into aggregate code generation.
{{AggImpState}} should receive the table from its callers, including
{{EnumerableAggregate}}, {{EnumerableSortedAggregate}}, {{EnumerableWindow}},
and the interpreter's {{AggregateNode}}, using
{{RexImplementorTables.of(getCluster())}}.
With this change, planning and code generation use the same implementor table.
If the enumerable aggregate rule accepts an aggregate, code generation can
resolve the same implementor.
{{RelOptCluster}} itself is not changed; it is only used to access the planner
{{Context}}.
h3. Backward Compatibility
If no custom table is registered, {{RexImplementorTables.of(RelOptCluster)}}
returns {{RexImpTable.instance()}}, so existing behavior is unchanged.
h2. Example
{code:sql}
SELECT g, MY_CUSTOM_AGG(x) AS c
FROM (VALUES ('a', 1), ('a', 2), ('b', 3)) AS t (g, x)
GROUP BY g
{code}
Here, {{MY_CUSTOM_AGG}} is contributed through a {{SqlOperatorTable}}, and its
implementation is provided by a chained {{RexImplementorTable}}. This pattern
can support custom aggregate functions such as {{APPROX_TOP_K}},
{{APPROX_COUNT_DISTINCT}}, and other approximate aggregates whose execution
strategy is provided by the implementor.
h2. Testing
End-to-end enumerable tests for a custom aggregate implementor, asserting the
actual result values.
The tests should cover:
* SQL execution through the {{Frameworks}} API.
* A directly built planner using a registered custom {{RexImplementorTable}}.
* A negative case where an unknown aggregate is rejected when no custom
implementor table is registered.
was:
h2. Background
CALCITE-7631 introduced {{RexImplementorTable}}, an interface for providing
implementors for custom functions defined through a {{SqlOperatorTable}}. As
part of that change, {{RexImpTable}} became the built-in implementation of
{{RexImplementorTable}}. The scalar code-generation path was updated to use
this composable table. {{RexToLixTranslator}}, and therefore
{{RexExecutorImpl}}, can now use a custom {{RexImplementorTable}} chained ahead
of the built-ins.
The aggregate path was not updated. {{EnumerableAggregate}} and related
aggregate code still resolve implementors directly from the {{RexImpTable}}
singleton. Therefore, custom aggregate operators contributed through a
{{SqlOperatorTable}} cannot be planned or executed by the enumerable engine
unless they are known to the built-in table.
h2. Problem
The aggregate path consults the built-in singleton in two places:
* *Planning time*: the {{EnumerableAggregate}} constructor checks each
{{AggregateCall}} against {{RexImpTable.INSTANCE}} and throws
{{InvalidRelException}} if no implementor is found. {{EnumerableAggregateRule}}
catches this and returns {{null}}, so aggregates unknown to the built-ins are
rejected during planning.
* *Code-generation time*: {{AggImpState}} also resolves aggregate implementors
from {{RexImpTable.INSTANCE}}.
Because both paths hard-code the singleton, a chained {{RexImplementorTable}}
containing custom aggregate implementors is ignored.
Implementor availability is a planner-dependent decision. It depends on the
active implementor table, which is available through the planner context, not
through the {{EnumerableAggregate}} constructor alone.
h2. Proposed Change
Make aggregate implementor resolution use the composable
{{RexImplementorTable}}, defaulting to the built-ins when no custom table is
registered.
* Add {{RexImplementorTables.of(RelOptCluster)}}. It returns the
{{RexImplementorTable}} registered in the planner {{Context}}, or
{{RexImpTable.instance()}} if none is registered.
* Move the aggregate implementor availability check from the
{{EnumerableAggregate}} constructor into {{EnumerableAggregateRule.convert}},
where the planner context is available. If the active table has no implementor,
the rule returns {{null}} and the planner may try another convention.
* Keep only table-independent structural validation in the
{{EnumerableAggregate}} constructor.
* Thread the resolved {{RexImplementorTable}} into aggregate code generation.
{{AggImpState}} should receive the table from its callers, including
{{EnumerableAggregate}}, {{EnumerableSortedAggregate}}, {{EnumerableWindow}},
and the interpreter's {{AggregateNode}}, using
{{RexImplementorTables.of(getCluster())}}.
With this change, planning and code generation use the same implementor table.
If the enumerable aggregate rule accepts an aggregate, code generation can
resolve the same implementor.
{{RelOptCluster}} itself is not changed; it is only used to access the planner
{{Context}}.
h3. Backward Compatibility
If no custom table is registered, {{RexImplementorTables.of(RelOptCluster)}}
returns {{RexImpTable.instance()}}, so existing behavior is unchanged.
h2. Example
{code:sql}
SELECT g, MY_CUSTOM_AGG(x) AS c
FROM (VALUES ('a', 1), ('a', 2), ('b', 3)) AS t (g, x)
GROUP BY g
{code}
Here, {{MY_CUSTOM_AGG}} is contributed through a {{SqlOperatorTable}}, and its
implementation is provided by a chained {{RexImplementorTable}}. This pattern
can support custom aggregate functions such as {{APPROX_TOP_K}},
{{APPROX_COUNT_DISTINCT}}, and other approximate aggregates whose execution
strategy is provided by the implementor.
h2. Testing
Add end-to-end enumerable tests for a custom aggregate implementor, asserting
the actual result values.
The tests should cover:
* SQL execution through the {{Frameworks}} API.
* A directly built planner using a registered custom {{RexImplementorTable}}.
* A negative case where an unknown aggregate is rejected when no custom
implementor table is registered.
> Enumerable engine should execute aggregates whose implementor comes from a
> custom RexImplementorTable"
> ------------------------------------------------------------------------------------------------------
>
> Key: CALCITE-7640
> URL: https://issues.apache.org/jira/browse/CALCITE-7640
> Project: Calcite
> Issue Type: Improvement
> Components: core
> Affects Versions: 1.42.0
> Reporter: Darpan Lunagariya (e6data computing)
> Assignee: Darpan Lunagariya (e6data computing)
> Priority: Minor
> Labels: pull-request-available
>
> h2. Background
> CALCITE-7631 introduced {{RexImplementorTable}}, an interface for providing
> implementors for custom functions defined through a {{SqlOperatorTable}}. As
> part of that change, {{RexImpTable}} became the built-in implementation of
> {{RexImplementorTable}}. The scalar code-generation path was updated to use
> this composable table. {{RexToLixTranslator}}, and therefore
> {{RexExecutorImpl}}, can now use a custom {{RexImplementorTable}} chained
> ahead of the built-ins.
> The aggregate path was not updated. {{EnumerableAggregate}} and related
> aggregate code still resolve implementors directly from the {{RexImpTable}}
> singleton. Therefore, custom aggregate operators contributed through a
> {{SqlOperatorTable}} cannot be planned or executed by the enumerable engine
> unless they are known to the built-in table.
> h2. Problem
> The aggregate path consults the built-in singleton in two places:
> * *Planning time*: the {{EnumerableAggregate}} constructor checks each
> {{AggregateCall}} against {{RexImpTable.INSTANCE}} and throws
> {{InvalidRelException}} if no implementor is found.
> {{EnumerableAggregateRule}} catches this and returns {{null}}, so aggregates
> unknown to the built-ins are rejected during planning.
> * *Code-generation time*: {{AggImpState}} also resolves aggregate
> implementors from {{RexImpTable.INSTANCE}}.
> Because both paths hard-code the singleton, a chained {{RexImplementorTable}}
> containing custom aggregate implementors is ignored.
> Implementor availability is a planner-dependent decision. It depends on the
> active implementor table, which is available through the planner context, not
> through the {{EnumerableAggregate}} constructor alone.
> h2. Proposed Change
> Make aggregate implementor resolution use the composable
> {{RexImplementorTable}}, defaulting to the built-ins when no custom table is
> registered.
> * Add {{RexImplementorTables.of(RelOptCluster)}}. It returns the
> {{RexImplementorTable}} registered in the planner {{Context}}, or
> {{RexImpTable.instance()}} if none is registered.
> * Move the aggregate implementor availability check from the
> {{EnumerableAggregate}} constructor into {{EnumerableAggregateRule.convert}},
> where the planner context is available. If the active table has no
> implementor, the rule returns {{null}} and the planner may try another
> convention.
> * Keep only table-independent structural validation in the
> {{EnumerableAggregate}} constructor.
> * Thread the resolved {{RexImplementorTable}} into aggregate code generation.
> {{AggImpState}} should receive the table from its callers, including
> {{EnumerableAggregate}}, {{EnumerableSortedAggregate}}, {{EnumerableWindow}},
> and the interpreter's {{AggregateNode}}, using
> {{RexImplementorTables.of(getCluster())}}.
> With this change, planning and code generation use the same implementor
> table. If the enumerable aggregate rule accepts an aggregate, code generation
> can resolve the same implementor.
> {{RelOptCluster}} itself is not changed; it is only used to access the
> planner {{Context}}.
> h3. Backward Compatibility
> If no custom table is registered, {{RexImplementorTables.of(RelOptCluster)}}
> returns {{RexImpTable.instance()}}, so existing behavior is unchanged.
> h2. Example
> {code:sql}
> SELECT g, MY_CUSTOM_AGG(x) AS c
> FROM (VALUES ('a', 1), ('a', 2), ('b', 3)) AS t (g, x)
> GROUP BY g
> {code}
> Here, {{MY_CUSTOM_AGG}} is contributed through a {{SqlOperatorTable}}, and
> its implementation is provided by a chained {{RexImplementorTable}}. This
> pattern can support custom aggregate functions such as {{APPROX_TOP_K}},
> {{APPROX_COUNT_DISTINCT}}, and other approximate aggregates whose execution
> strategy is provided by the implementor.
> h2. Testing
> End-to-end enumerable tests for a custom aggregate implementor, asserting the
> actual result values.
> The tests should cover:
> * SQL execution through the {{Frameworks}} API.
> * A directly built planner using a registered custom {{RexImplementorTable}}.
> * A negative case where an unknown aggregate is rejected when no custom
> implementor table is registered.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
