Hi,
Apache Calcite has a powerful but complicated and non-documented function
library. Some projects may require overriding some of the existing
operators to introduce custom type deduction, custom validation, etc. This
includes the base arithmetic functions (e.g, disallow INT + VARCHAR),
aggregate functions (e.g., custom precision extension), etc.
One convenient way of doing this is to re-define the function in your
custom operator table. However, this doesn't work because Apache Calcite
core uses the direct references to SqlStdOperatorTable. It starts with the
parser [1] and validator [2]. If you manage to inject your functions at
this stage (e.g., using a custom validator implementation or a custom
SqlVisitor), the sql-to-rel converter will overwrite your functions
still [3]. And even when you get the RelNode, optimization rules would
silently replace your custom functions with the default ones [4].
Alternatively, you may try extending some base interface, such as the
TypeCoercion, but this doesn't give fine-grained control over the function
behavior because you have to retain the existing function definitions to do
coercion works.
A better solution might be is to abstract out the function references
through some sort of "factory"/"resolver", somewhat similar to the one used
to resolve user-provided operators. For instance, the user may pass an
optional desired operator table to parser/validator/converter configs and
RelOptCluster. Then the "core" codebase could be refactored to dereference
functions by SqlKind instead of SqlStdOperatorTable. If some required
function types are missing from the SqlKind enum, we can add them. The
default behavior would delegate to SqlStdOperatorTable, so the existing
apps would not be affected.
A "small" problem is that there are ~1500 usages of the SqlStdOperatorTable
in the "core" module, but most of the usages are very straightforward to
replace.
This way, we would ensure a consistent function resolution throughout all
query optimization phases. WDYT?
Regards,
Vladimir.
[1]
https://github.com/apache/calcite/blob/calcite-1.30.0/core/src/main/codegen/templates/Parser.jj#L7164
[2]
https://github.com/apache/calcite/blob/calcite-1.30.0/core/src/main/java/org/apache/calcite/sql/validate/SqlValidatorImpl.java#L6788
[3]
https://github.com/apache/calcite/blob/calcite-1.30.0/core/src/main/java/org/apache/calcite/sql2rel/SqlToRelConverter.java#L1438
[4]
https://github.com/apache/calcite/blob/calcite-1.30.0/core/src/main/java/org/apache/calcite/rel/rules/AggregateReduceFunctionsRule.java#L357
