Hello Fang-Yu Rao, Daniel Becker, Csaba Ringhofer, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19429
to look at the new patch set (#2).
Change subject: IMPALA-11845: Fix incorrect check of struct STAR path in
resolvePathWithMasking
......................................................................
IMPALA-11845: Fix incorrect check of struct STAR path in resolvePathWithMasking
resolvePathWithMasking() is a wrapper on resolvePath() to further
resolve nested columns inside the table masking view. When it was
added, complex types in the select list hadn't been supported yet. So
the table masking view can't expose complex type columns directly in the
select list. Any paths in nested types will be further resolved inside
the table masking view in resolvePathWithMasking().
Take the following query as an example:
select id, nested_struct.* from complextypestbl;
If Ranger column-masking/row-filter policies applied on the table, the
query is rewritten as
select id, nested_struct.* from (
select mask(id) from complextypestbl
where row-filtering-condition
) t;
Table masking view "t" can't expose the nested column "nested_struct".
So we further resolve "nested_struct" inside the inlineView to use the
masked table "complextypestbl". The underlying TableRef is expected to
be a BaseTableRef.
Paths that don't reference nested columns should be resolved and
returned directly (just like the original resolvePath() does). E.g.
select v.* from masked_view v
is rewritten to
select v.* from (
select mask(c1), mask(c2), ..., mask(cn)
from masked_view
where row-filtering-condition
) v;
The STAR path "v.*" should be resolved directly. However, it's treated as a
nested column unexpectedly. The code then try to resolve it inside the
table "masked_view" and found it's not a table so throws the
IllegalStateException.
These are the conditions for returning the STAR paths directly:
- The type is STRUCT
- And the resolved path is rooted at a valid tuple descriptor
They don't really recognize the nested columns. STAR expansion is only
valid for paths to a struct type (or a table/view). So the first
condition always matches. The second condition also matches for STAR
paths on table/view, i.e. paths of "v.*" when "v" is a catalog
table/view. The rooted tuple descriptor is exactly the output tuple of
the table/view.
This patch fixes the check for nested struct STAR expansion by checking
the matched types instead. Note that if "v.*" is a table/view expansion,
the matched type list is empty. If "v.*" is a struct column expansion,
the matched type list contains the STRUCT column type.
Tests:
- Add missing coverage on STAR paths (v.*) on masked views.
Change-Id: I8f1e78e325baafbe23101909d47e82bf140a2d77
---
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/Path.java
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking.test
M
testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test
4 files changed, 67 insertions(+), 4 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/19429/2
--
To view, visit http://gerrit.cloudera.org:8080/19429
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I8f1e78e325baafbe23101909d47e82bf140a2d77
Gerrit-Change-Number: 19429
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com>
Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com>
Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com>
