GitHub user KulykRoman opened a pull request:
https://github.com/apache/drill/pull/1066
DRILL-3993: Changes to support Calcite 1.13
Works with Drill-specific Calcite 1.13 from a branch:
https://github.com/KulykRoman/incubator-calcite/tree/DRILL-3993.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/KulykRoman/drill DRILL-3993
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/drill/pull/1066.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1066
----
commit 71d8b03d60dcbe4d46f1f12aba82354064c25ef6
Author: Roman Kulyk <[email protected]>
Date: 2017-11-30T16:19:12Z
REVERTED: DRILL-5089
Dynamically load schema of storage plugin only when needed for every query
commit 1c3cea13c798eb831f9e9911ec144bf4824ac5ad
Author: Roman Kulyk <[email protected]>
Date: 2017-08-29T14:10:24Z
DRILL-3993: Changes to support Calcite 1.13
- fixed all compiling errors (main changes were: Maven changes, chenges
RelNode -> RelRoot, implementing some new methods from updated interfaces,
chenges some literals, logger changes);
- fixed unexpected column errors, validation errors and assertion errors
after Calcite update;
- fixed describe table/schema statement according to updated logic;
- added fixes with time-intervals;
- changed precision of BINARY to 65536 (was 1048576) according to updated
logic (Calcite overrides bigger precision to own maxPrecision);
- ignored some incorrect tests with DRILL-3244;
- changed "Table not found" message to "Object not found within" according
to new Calcite changes.
commit 7196b366994f6ebcdcb80ca07701e06955e2b856
Author: Volodymyr Vysotskyi <[email protected]>
Date: 2017-11-03T12:18:09Z
DRILL-3993: Fix unit test failures connected with support Calcite 1.13
- Use root schema as default for describe table statement.
Fix TestOpenTSDBPlugin.testDescribe() and
TestInfoSchemaOnHiveStorage.varCharMaxLengthAndDecimalPrecisionInInfoSchema()
unit tests.
- Modify expected results for tests:
TestPreparedStatementProvider.invalidQueryValidationError();
TestProjectPushDown.testTPCH1();
TestProjectPushDown.testTPCH3();
TestStorageBasedHiveAuthorization.selectUser1_db_u0_only();
TestStorageBasedHiveAuthorization.selectUser0_db_u1g1_only()
- Fix TestCTAS.whenTableQueryColumnHasStarAndTableFiledListIsSpecified(),
TestViewSupport.createViewWhenViewQueryColumnHasStarAndViewFiledListIsSpecified(),
TestInbuiltHiveUDFs.testIf(), testDisableUtf8SupportInQueryString unit tests.
- Fix UnsupportedOperationException and NPE for jdbc tests.
- Fix AssertionError: Conversion to relational algebra failed to preserve
datatypes
*DrillCompoundIdentifier:
According to the changes, made in [CALCITE-546], star Identifier is
replaced by empty string during parsing the query. Since Drill uses its own
DrillCompoundIdentifier, it should also replace star by empty string before
creating SqlIdentifier instance to avoid further errors connected with star
column. see SqlIdentifier.isStar() method.
*SqlConverter:
In [CALCITE-1417] added simplification of expressions which should be
projected every time when a new project rel node is created using RelBuilder.
It causes assertion errors connected with types nullability. This hook was set
to false to avoid project expressions simplification. See usage of this hook
and RelBuilder.project() method.
In Drill the type nullability of the function depends on only the
nullability of its arguments. In some cases, a function may return null value
even if it had non-nullable arguments. When Calice simplifies expressions, it
checks that the type of the result is the same as the type of the expression.
Otherwise, makeCast() method is called. But when a function returns null
literal, this cast does nothing, even when the function has a non-nullable
type. So to avoid this issue, method makeCast() was overridden.
*DrillAvgVarianceConvertlet:
Problem with sum0 and specific changes in old Calcite (it is CALCITE-777).
(see HistogramShuttle.visitCall method) Changes were made to avoid changes in
Calcite.
*SqlConverter, DescribeTableHandler, ShowTablesHandler:
New Calcite tries to combine both default and specified workspaces during
the query validation. In some cases, for example, when describe table statement
is used, Calcite tries to find INFORMATION_SCHEMA in the schema used as
default. When it does not find the schema, it tries to find a table with such
name. For some storage plugins, such as opentsdb and hbase, when a table was
not found, the error is thrown, and the query fails. To avoid this issue,
default schema was changed to root schema for validation stage for describe
table and show tables queries.
commit 56f2c9ac47d224e2ac02ab5036e7aa01df1951f2
Author: Volodymyr Vysotskyi <[email protected]>
Date: 2017-11-15T10:37:50Z
DRILL-3993: Use custom RelBuilder implementation in rules
After the changes, made in CALCITE-1056 if the filter has a predicate that
is always false, RelBuilder.filter() method returns values rel node instead of
filter rel node. In order to preserve column types, DrillRelBuilder.empty()
method, which is returned by filter method was overridden, and now it returns
filter with a false predicate. (advice to override this method was in its
javadoc) The goal of all other changes in this commit is to use our custom
RelBuilder for all rules that are used in Drill.
commit 82c750aee2ad67a1e9e0227ca06d39f9362cba99
Author: Roman Kulyk <[email protected]>
Date: 2017-11-02T18:22:36Z
DRILL-3993: Fix failed tests after Calcite update
- fix temporary table errors according to updated logic;
- fixed errors when we trying to make select from hbase table with schema
name in query (example: "SELECT row_key FROM hbase.TestTableNullStr) from hbase
schema (did "USE hbase" before). Added test for it;
- added fix for views which were created on Calcite 1.4 and test for it.
----
---
