Tamas Mate has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/21061 )
Change subject: IMPALA-12610: Support reading ARRAY columns for Iceberg Metadata tables ...................................................................... IMPALA-12610: Support reading ARRAY columns for Iceberg Metadata tables This commit adds support for reading ARRAY columns inside Iceberg Metadata tables. The change starts with some refactoring, to consolidate accessing JVM through JNI a new backend class was introduced, IcebergMetadataScanner. This class is the C++ part of the Java IcebergMetadataScanner, it is responsible to manage the Java scanner object. In Iceberg array types do not have accessors, so structs inside arrays have to be accessed by position, for the value obtaining logics have been changed to allow access by position. The IcebergRowReader needed an IcebergMetadataScanner, so that it can iterate over the arrays returned by the scanner and add them to the collection. This change will not cover MAP, these slots are set to NULL, it will be done in IMPALA-12611. Testing: - Added E2E tests. Change-Id: Ieb9bac1822a17bd3cd074b4b98e4d010703cecb1 --- M be/src/exec/iceberg-metadata/CMakeLists.txt M be/src/exec/iceberg-metadata/iceberg-metadata-scan-node.cc M be/src/exec/iceberg-metadata/iceberg-metadata-scan-node.h A be/src/exec/iceberg-metadata/iceberg-metadata-scanner.cc A be/src/exec/iceberg-metadata/iceberg-metadata-scanner.h M be/src/exec/iceberg-metadata/iceberg-row-reader.cc M be/src/exec/iceberg-metadata/iceberg-row-reader.h M be/src/service/impalad-main.cc M fe/src/main/java/org/apache/impala/analysis/FromClause.java M fe/src/main/java/org/apache/impala/util/IcebergMetadataScanner.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test 11 files changed, 734 insertions(+), 292 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/21061/10 -- To view, visit http://gerrit.cloudera.org:8080/21061 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ieb9bac1822a17bd3cd074b4b98e4d010703cecb1 Gerrit-Change-Number: 21061 Gerrit-PatchSet: 10 Gerrit-Owner: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Tamas Mate <tma...@apache.org> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
