Hello Joe McDonnell,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21195
to look at the new patch set (#9).
Change subject: IMPALA-12935: First pass on Calcite planner functions
......................................................................
IMPALA-12935: First pass on Calcite planner functions
This commit handles the first pass on getting functions to work
through the Calcite planner. Only basic functions will work with
this commit. Implicit conversions for parameters are not yet supported.
Custom UDFs are also not supported yet.
The functions are loaded in CalciteJniFrontend from the Impala "builtin"
database. A "FunctionSignature" is created for each builtin function.
Each function is loaded into Calcite's "ImpalaOperatorTable" object which
is used in the validation process. Each Impala function maps to a Calcite
"ImpalaOperator" object.
For function names that are not an exact match with a Calcite operator,
an entry is found in FunctionDetailStatics to do the conversion.
When the Calcite validator tries to validate the function, it gets the
return type through the "ImpalaOperator.inferReturnType()" method. In
this method, a "FunctionSignatureForLookup" signature is created for
looking up the stored signatures. The "Lookup" signature does not have
a return type (yet), so the matching of signatures will be based on
matching name and parameters. As mentioned above, implicit conversion
is not yet supported in this commit; this will be added later.
After validation is complete, the functions will be in a Calcite format.
After the rest of compilation (relnode conversion, optimization) is
complete, the function needs to be converted back into Impala form (the
Expr object) to eventually get it into its thrift request.
In this commit, all functions are converted into Expr starting in the
ImpalaProjectRel, since this is the RelNode where functions do their
thing. The RexCallConverter and RexLiteralConverter get called via the
CreateExprVisitor for this conversion.
Since Calcite is providing the analysis portion of the planning, there
is no need to go through Impala's Analyzer object. However, the Impala
planner requires Expr objects to be analyzed. To get around this, the
AnalyzedFunctionCallExpr and AnalyzedNullLiteral objects exist which
analyze the expression in the constructor. While this could potentially
be combined with the existing FunctionCallExpr and NullLiteral objects,
this fits in with the general plan to avoid changing "fe" Impala code
as much as we can until much later in the commit cycle. Also, there
will be other Analyzed*Expr classes created in the future, but this
commit is intended for basic function call expressions only.
One minor change to the parser is added with this commit. Calcite parser
does not have acknowledge the "string" datatype, so this has been
added here in Parser.jj and config.fmpp.
Change-Id: If8b57fab2a6b422e4a76b7b4d70fb75bbe0b81ad
---
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionDetailStatics.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignature.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignatureForLookup.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignatureForStorage.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaAggregateOperator.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaHelperOperator.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperatorTable.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaScalarOperator.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/parserimpl/codegen/config.fmpp
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/parserimpl/codegen/templates/Parser.jj
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
M testdata/workloads/functional-query/queries/QueryTest/calcite.test
21 files changed, 1,780 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/21195/9
--
To view, visit http://gerrit.cloudera.org:8080/21195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: If8b57fab2a6b422e4a76b7b4d70fb75bbe0b81ad
Gerrit-Change-Number: 21195
Gerrit-PatchSet: 9
Gerrit-Owner: Steve Carlin <scar...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
