Hello Aman Sinha, Joe McDonnell, Csaba Ringhofer, Michael Smith, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21357
to look at the new patch set (#4).
Change subject: IMPALA-12935: First pass on Calcite planner functions
......................................................................
IMPALA-12935: First pass on Calcite planner functions
This commit handles the first pass on getting functions to work
through the Calcite planner. Only basic functions will work with
this commit. Implicit conversions for parameters are not yet supported.
Custom UDFs are also not supported yet.
The functions are loaded in CalciteJniFrontend from the Impala "builtin"
database. A "FunctionSignature" is created for each builtin function.
Each function is loaded into Calcite's "ImpalaOperatorTable" object which
is used in the validation process. Each Impala function maps to a Calcite
"ImpalaOperator" object.
For function names that are not an exact match with a Calcite operator,
an entry is found in FunctionDetailStatics to do the conversion.
When the Calcite validator tries to validate the function, it gets the
return type through the "ImpalaOperator.inferReturnType()" method. In
this method, a "FunctionSignatureForLookup" signature is created for
looking up the stored signatures. The "Lookup" signature does not have
a return type (yet), so the matching of signatures will be based on
matching name and parameters. As mentioned above, implicit conversion
is not yet supported in this commit; this will be added later.
After validation is complete, the functions will be in a Calcite format.
After the rest of compilation (relnode conversion, optimization) is
complete, the function needs to be converted back into Impala form (the
Expr object) to eventually get it into its thrift request.
In this commit, all functions are converted into Expr starting in the
ImpalaProjectRel, since this is the RelNode where functions do their
thing. The RexCallConverter and RexLiteralConverter get called via the
CreateExprVisitor for this conversion.
Since Calcite is providing the analysis portion of the planning, there
is no need to go through Impala's Analyzer object. However, the Impala
planner requires Expr objects to be analyzed. To get around this, the
AnalyzedFunctionCallExpr and AnalyzedNullLiteral objects exist which
analyze the expression in the constructor. While this could potentially
be combined with the existing FunctionCallExpr and NullLiteral objects,
this fits in with the general plan to avoid changing "fe" Impala code
as much as we can until much later in the commit cycle. Also, there
will be other Analyzed*Expr classes created in the future, but this
commit is intended for basic function call expressions only.
One minor change to the parser is added with this commit. Calcite parser
does not have acknowledge the "string" datatype, so this has been
added here in Parser.jj and config.fmpp.
Change-Id: I2dd4e402d69ee10547abeeafe893164ffd789b88
---
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedNullLiteral.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionDetailStatics.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignature.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignatureForLookup.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionSignatureForStorage.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaAggregateOperator.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaHelperOperator.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaOperatorTable.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/operators/ImpalaScalarOperator.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/parserimpl/codegen/config.fmpp
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/parserimpl/codegen/templates/Parser.jj
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/util/CreateExprVisitor.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteJniFrontend.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteValidator.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
M testdata/workloads/functional-query/queries/QueryTest/calcite.test
21 files changed, 1,740 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/21357/4
--
To view, visit http://gerrit.cloudera.org:8080/21357
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2dd4e402d69ee10547abeeafe893164ffd789b88
Gerrit-Change-Number: 21357
Gerrit-PatchSet: 4
Gerrit-Owner: Steve Carlin <scar...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
