Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23383
to look at the new patch set (#10).
Change subject: IMPALA-14295: [WIP] Support table masking for the Calcite
planner
......................................................................
IMPALA-14295: [WIP] Support table masking for the Calcite planner
This patch only deals with column masking at the moment.
Unlike https://gerrit.cloudera.org/c/23383/9, in this patch, we do not
replace an AST node that corresponds to a table that has to be masked.
Instead, we create a masked validated SqlNode representing the masked
table and then add it to ImpalaCalciteCatalogReader during query
validation. Later on, during the AST-to-logical plan conversion when
getTable(List<String> names) is called, ImpalaCalciteCatalogReader first
checks whether there is already a masked table identified by the same
name. If so, we return a RelOptTableImpl that could later be expanded,
and we return super.getTable(names) otherwise.
Testing:
- Verified that it could execute the following queries when there are
the following policies defined.
Policy 1:
a column masking policy defined on the column 'id' of the
table 'functional.alltypes' (using an expression like 'id * 111')
Policy 2:
another column masking policy defined on the column
'bigint_col' of the same table (using an expression like
'((select count(bool_col) from functional.alltypestiny))' that
references another table 'alltypestiny'.
Query 1: query against a table where there are 2 column masking
policies defined
select bigint_col, id from functional.alltypes order by id limit 3
Query 2: join with another table 'alltypestiny'
select functional.alltypes.id,
functional.alltypes.string_col from
functional.alltypes, functional.alltypestiny where
functional.alltypes.string_col = functional.alltypestiny.string_col
order by functional.alltypes.id
limit 10
Query 3: query against a view defined on top of a table against which
there are column masking policies defined
select bigint_col, id from functional.alltypes_view order by id
limit 3
To-do's:
- Clean up the unused code and incorrect code comments.
- To see if we need to create a new class other than ImpalaViewTable
when storing the validated masked table in
ImpalaCalciteCatalogReader. Refer to applyTableMaskingIfNeeded().
- Need to register the privilege requests for the tables and columns
used in a table masking policy if a referenced table is NOT the
masked table, e.g., the privilege requests for the table
'alltypestiny' and its column 'bool_col' in the 2nd policy.
- Test with more complicated cases, e.g., when there are table masking
policies defined on regular/catalog views.
Change-Id: I3d9cf028078762f78cf0f82b817bdc1ee37e180b
---
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaCalciteCatalogReader.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaViewTable.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlToRelConverter.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ImpalaSqlValidatorImpl.java
7 files changed, 536 insertions(+), 8 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/23383/10
--
To view, visit http://gerrit.cloudera.org:8080/23383
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3d9cf028078762f78cf0f82b817bdc1ee37e180b
Gerrit-Change-Number: 23383
Gerrit-PatchSet: 10
Gerrit-Owner: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Fang-Yu Rao <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
