Hello Tamas Mate, Gabor Kaszab, lipeng...@sensorsdata.cn, Gergely Fürnstáhl,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/18847
to look at the new patch set (#10).
Change subject: IMPALA-11484: Create SCAN plan for Iceberg V2 position delete
tables
......................................................................
IMPALA-11484: Create SCAN plan for Iceberg V2 position delete tables
This patch adds support for reading Iceberg V2 tables use position
deletes. Equality deletes are still not supported. Position delete
files store the file path and file position of the deleted rows.
When an Iceberg table has position delete files we need to do an
ANTI JOIN between data files and delete files. From the data files
we need to query the virtual columns INPUT__FILE__NAME and
FILE__POSITION, while from the delete files we need the data columns
'file_path' and 'pos'. The latter data columns are not part of the
table schema, so we create a virtual table instance of
'IcebergPositionDeleteTable' that has a table schema corresponding
to the delete files ('file_path', 'pos').
This patch introduces a new class 'IcebergScanPlanner' which has
the responsibility of doing a plan for Iceberg table scans. It creates
the aforementioned ANTI JOIN. Also, if there are data files without
corresponding delete files, we can have a separate SCAN node and its
results would be UNIONed to the rows coming from the ANTI JOIN:
UNION
/ \
SCAN data ANTI JOIN
/ \
SCAN data SCAN deletes
Some refactorings in the context of this CR:
Predicate pushdown and time travel logic is transferred from
IcebergScanNode to IcebergScanPlanner. Iceberg snapshot summary
retrieval is moved from FeFsTable to FeIcebergTable.
Testing:
* added planner test
* added e2e tests
TODO in follow-up Jiras:
* better cardinality estimates (IMPALA-11516)
* support unrelative collection columns (select item from t.int_array)
(IMPALA-11517)
Currently such queries return error during analysis
Change-Id: I672cfee18d8e131772d90378d5b12ad4d0f7dd48
---
M common/thrift/CatalogObjects.thrift
M fe/src/main/java/org/apache/impala/analysis/AlterTableSetTblProperties.java
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/main/java/org/apache/impala/analysis/SelectStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
M fe/src/main/java/org/apache/impala/catalog/FeFsTable.java
M fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/IcebergPositionDeleteTable.java
M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java
A fe/src/main/java/org/apache/impala/catalog/VirtualTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalIcebergTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java
M fe/src/main/java/org/apache/impala/planner/HashJoinNode.java
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/IcebergScanNode.java
A fe/src/main/java/org/apache/impala/planner/IcebergScanPlanner.java
M fe/src/main/java/org/apache/impala/planner/JoinNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M fe/src/main/java/org/apache/impala/planner/UnionNode.java
M fe/src/main/java/org/apache/impala/util/IcebergUtil.java
M fe/src/test/java/org/apache/impala/planner/PlannerTest.java
M testdata/data/README
M
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_delete_positional/data/00191-4-6e780302-527b-4911-8c6e-88d416adac57-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/data/00000-0-data-boroknagyz_20220819180420_a7e5a731-8762-4b59-b3f2-fe6f065cf59b-job_16597105613620_0031-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/5c80922f-01b5-4d52-bc93-6505be3b977b-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/snap-728158873687794725-1-5c80922f-01b5-4d52-bc93-6505be3b977b.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/data/00000-0-data-boroknagyz_20220830153046_7c9aaa51-d601-4c5b-a51c-67c1e9e1fe29-job_16618645111570_0001-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/a72290c9-c518-4719-8502-6c83a881de07-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/snap-1041485290740594175-1-a72290c9-c518-4719-8502-6c83a881de07.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_no_deletes_orc/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=click/00000-0-delete-boroknagyz_20220819183231_cfc565f5-52b9-4669-9f69-d29c50a84a5e-job_16597105613621_0032-00002.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=click/874b32d9a15da206-f60e01cb00000003_1034098606_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=download/00000-0-delete-boroknagyz_20220819183231_cfc565f5-52b9-4669-9f69-d29c50a84a5e-job_16597105613621_0032-00003.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=download/874b32d9a15da206-f60e01cb00000003_1489587766_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=view/00000-0-delete-boroknagyz_20220819183231_cfc565f5-52b9-4669-9f69-d29c50a84a5e-job_16597105613621_0032-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/data/action=view/874b32d9a15da206-f60e01cb00000004_1711435901_data.0.parq
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/464c179e-c9ba-40f5-a35f-144106a1f16c-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/771485e9-78ac-4ffc-b1ef-1fda5bab33cf-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/snap-2057976186205897384-1-771485e9-78ac-4ffc-b1ef-1fda5bab33cf.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/snap-8885697082976537578-1-464c179e-c9ba-40f5-a35f-144106a1f16c.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=click/00000-0-data-boroknagyz_20220830173852_560d6f0e-8508-42d6-95a7-67bfa860a28c-job_16618645111571_0006-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=click/00000-0-delete-boroknagyz_20220830173925_010bf879-cf2f-4209-a70b-1a8309657106-job_16618645111571_0006-00002.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=download/00000-0-data-boroknagyz_20220830173852_560d6f0e-8508-42d6-95a7-67bfa860a28c-job_16618645111571_0006-00002.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=download/00000-0-delete-boroknagyz_20220830173925_010bf879-cf2f-4209-a70b-1a8309657106-job_16618645111571_0006-00003.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=view/00000-0-data-boroknagyz_20220830173852_560d6f0e-8508-42d6-95a7-67bfa860a28c-job_16618645111571_0006-00003.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/data/action=view/00000-0-delete-boroknagyz_20220830173925_010bf879-cf2f-4209-a70b-1a8309657106-job_16618645111571_0006-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/588ee1ca-6a85-4af2-8ba2-e595e71712ba-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/5b14eaa8-83cb-4f71-a473-402f345fa5b5-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/snap-5359840930115020310-1-5b14eaa8-83cb-4f71-a473-402f345fa5b5.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/snap-5416468273053855108-1-588ee1ca-6a85-4af2-8ba2-e595e71712ba.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_partitioned_position_deletes_orc/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/data/00000-0-data-boroknagyz_20220819150927_e95cefe8-5549-42f6-98a6-b1bbea9a4224-job_16597105613620_0025-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/data/00000-0-delete-boroknagyz_20220819150939_8f0789df-fcdb-4ba8-875c-4aeac53f30b9-job_16597105613621_0025-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/236523f7-a5bc-459f-b4c9-16af5bd43bca-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/816400dd-012d-40c5-ab65-bc16ff18d2d7-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/snap-444149380144800647-1-816400dd-012d-40c5-ab65-bc16ff18d2d7.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/snap-8593920101374128463-1-236523f7-a5bc-459f-b4c9-16af5bd43bca.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/data/00000-0-data-boroknagyz_20220830153712_a4060925-000f-4454-b68b-8f7570d3aa7c-job_16618645111570_0002-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/data/00000-0-delete-boroknagyz_20220830153733_8f03a46b-e4a5-4643-896b-a1cc5b09cf2b-job_16618645111571_0002-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/6d54c2e9-a4c7-4c5c-8a6f-a17f92de4c48-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/83d17f01-336e-41ab-a791-ffd5f511f6ab-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/snap-1801547319505512253-1-6d54c2e9-a4c7-4c5c-8a6f-a17f92de4c48.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/snap-4807054508647143162-1-83d17f01-336e-41ab-a791-ffd5f511f6ab.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_delete_all_rows_orc/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-data-boroknagyz_20220819154646_1cad8c38-c65e-4c7c-b516-c4d9faf82448-job_16597105613620_0026-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-data-boroknagyz_20220819154718_db95aeae-d530-4fba-8336-e47fa712b987-job_16597105613620_0026-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-data-boroknagyz_20220819154733_162cd69d-80e2-425b-9638-9008d0937573-job_16597105613620_0026-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-data-boroknagyz_20220819154922_62429d29-6c44-4707-b348-ac189b8d79d3-job_16597105613620_0026-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-delete-boroknagyz_20220819154922_62429d29-6c44-4707-b348-ac189b8d79d3-job_16597105613621_0026-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/data/00000-0-delete-boroknagyz_20220819155000_b0411d64-66e3-49fd-a2f0-dab69282a896-job_16597105613621_0027-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/4ba5ef32-7f8b-4418-b5bf-3fb8002e0dde-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/969de65c-8915-4ae5-8d54-a82701195c55-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/c4e37595-32c2-483f-b6d7-866f60e36976-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/db6f17fe-6fb6-4120-839d-4d6ca5244a1c-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/ea749da1-7b98-4dca-a4eb-f7d5d62f9dde-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/ec9e7ecc-b546-42dd-8d0d-0dde2182dbc7-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-1497619269847778439-1-ea749da1-7b98-4dca-a4eb-f7d5d62f9dde.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-4363979609026842966-1-db6f17fe-6fb6-4120-839d-4d6ca5244a1c.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-5762682948883272650-1-969de65c-8915-4ae5-8d54-a82701195c55.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-7490459762454857930-1-ec9e7ecc-b546-42dd-8d0d-0dde2182dbc7.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-7508485421322116327-1-4ba5ef32-7f8b-4418-b5bf-3fb8002e0dde.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/snap-752781918366351945-2-c4e37595-32c2-483f-b6d7-866f60e36976.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v4.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v5.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v6.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/v7.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-data-boroknagyz_20220830155232_f36b3f3e-183a-475a-a7d1-d16898fd4524-job_16618645111570_0003-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-data-boroknagyz_20220830155307_610e0b78-1186-477b-a30b-9943cb4e90e9-job_16618645111570_0003-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-data-boroknagyz_20220830155337_fcbda57d-f37d-48cd-afe4-28d0bad0a4c0-job_16618645111570_0004-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-data-boroknagyz_20220830155411_eee97f13-df3d-408c-bcff-8e3b0dcf8fa7-job_16618645111570_0004-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-delete-boroknagyz_20220830155337_fcbda57d-f37d-48cd-afe4-28d0bad0a4c0-job_16618645111571_0004-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/data/00000-0-delete-boroknagyz_20220830155430_32c848c9-37ad-4f77-a0fd-0a254d15246c-job_16618645111571_0004-00001.orc
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/560427d8-9e81-4111-9b40-3aeb73af7049-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/5f58eff5-d9a4-44eb-a208-244d0b6b927b-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/666190a5-67e0-431f-b142-7ddf9f933d4f-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/9677edab-12db-429c-aa24-5e3a112caed4-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/a8152ff6-a39c-435a-916b-d0df02388331-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/e1f383b4-0b8a-4a19-8c70-4ea23978309d-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-5003445199566617082-1-5f58eff5-d9a4-44eb-a208-244d0b6b927b.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-5936629849848085185-1-e1f383b4-0b8a-4a19-8c70-4ea23978309d.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-8476486151350891395-1-9677edab-12db-429c-aa24-5e3a112caed4.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-8479298697893993480-1-560427d8-9e81-4111-9b40-3aeb73af7049.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-8621312932233473501-1-666190a5-67e0-431f-b142-7ddf9f933d4f.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/snap-8634123667305136474-2-a8152ff6-a39c-435a-916b-d0df02388331.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v4.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v5.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v6.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/v7.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_not_all_data_files_have_delete_files_orc/metadata/version-hint.txt
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/data/00000-0-data-boroknagyz_20220819155751_2ddc0706-4c90-42db-927b-d92f68b8ae1e-job_16597105613620_0030-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/data/00000-0-data-boroknagyz_20220819155811_bf677db1-cb6c-451b-afc8-4013ae7eaf1f-job_16597105613620_0030-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/data/00000-0-delete-boroknagyz_20220819155811_bf677db1-cb6c-451b-afc8-4013ae7eaf1f-job_16597105613621_0030-00001.parquet
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/0f93551b-cc05-4e55-a65b-6f78e634cf4c-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/d399b245-f138-4ae2-bd1f-f82e0ee16023-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/da27d446-94df-4850-85dd-c8edcda7685e-m0.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/snap-3877007445826010687-2-da27d446-94df-4850-85dd-c8edcda7685e.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/snap-5392552459484846077-1-d399b245-f138-4ae2-bd1f-f82e0ee16023.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/snap-6537343489442250314-1-0f93551b-cc05-4e55-a65b-6f78e634cf4c.avro
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/v1.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/v2.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/v3.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/v4.metadata.json
A
testdata/data/iceberg_test/hadoop_catalog/ice/iceberg_v2_positional_update_all_rows/metadata/version-hint.txt
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
A
testdata/workloads/functional-planner/queries/PlannerTest/iceberg-v2-tables.test
M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes-orc.test
A
testdata/workloads/functional-query/queries/QueryTest/iceberg-v2-read-position-deletes.test
M tests/query_test/test_iceberg.py
156 files changed, 5,942 insertions(+), 555 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/47/18847/10
--
To view, visit http://gerrit.cloudera.org:8080/18847
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I672cfee18d8e131772d90378d5b12ad4d0f7dd48
Gerrit-Change-Number: 18847
Gerrit-PatchSet: 10
Gerrit-Owner: Zoltan Borok-Nagy <borokna...@cloudera.com>
Gerrit-Reviewer: Anonymous Coward <lipeng...@sensorsdata.cn>
Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gfurnst...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tma...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
