Hello Tamas Mate, Gabor Kaszab, Zoltan Borok-Nagy, Gergely Fürnstáhl, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19380 to look at the new patch set (#7). Change subject: IMPALA-10893: Use old schema during iceberg time travel. ...................................................................... IMPALA-10893: Use old schema during iceberg time travel. Before this change the schema used during Iceberg Time Travel was the current schema of the table. With this change we will use the schema from the point specified by the Time Travel parameters. The parameters used by an Iceberg Time Travel query are part of the FROM clause of the query. Previously analysis of the Time Travel parameters took place after the table Path was resolved, at which point some schema information is cached. In order to use the old schema during iceberg time travel however we need to ensure that the version of the Table that is used is always the version specified by the Time Travel parameters. To do this we have to move the analysis of the Time Travel parameters inside the code that resolves the Path. Add a new implementation of FeIcebergTable that represents an Iceberg table involved in Time Travel. This is implemented by embedding a reference to the base Iceberg Table. All methods that are not Time Travel related are delegated to the base table. The Time Travel related methods use the historic Iceberg schema. TESTING: - Add a new file iceberg_util.py to hold the snapshot utility code that was developed for the in-progress IMPALA-11482. - Extend the existing Iceberg Time Travel tests to check the schema. - Add a test that shows time travel working with columns masking. The column masking configuration is not tightly coupled to the schema so it is possible to mask historical columns. Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/DescriptorTable.java M fe/src/main/java/org/apache/impala/analysis/TableRef.java M fe/src/main/java/org/apache/impala/analysis/TimeTravelSpec.java M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java A fe/src/main/java/org/apache/impala/catalog/IcebergTimeTravelTable.java M fe/src/main/java/org/apache/impala/catalog/iceberg/IcebergCtasTarget.java M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M tests/authorization/test_ranger.py M tests/common/iceberg_test_suite.py M tests/query_test/test_iceberg.py A tests/util/iceberg_util.py 16 files changed, 1,036 insertions(+), 98 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/19380/7 -- To view, visit http://gerrit.cloudera.org:8080/19380 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I7cbef6e20bbb567e517744fb1f34d880970399ab Gerrit-Change-Number: 19380 Gerrit-PatchSet: 7 Gerrit-Owner: Andrew Sherman <asher...@cloudera.com> Gerrit-Reviewer: Andrew Sherman <asher...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>