Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/21125 )
Change subject: IMPALA-12611: Add support to MAP type Iceberg Metadata table columns ...................................................................... Patch Set 6: (8 comments) http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-metadata-scanner.h File be/src/exec/iceberg-metadata/iceberg-metadata-scanner.h: http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-metadata-scanner.h@63 PS4, Line 63: /// Note that it returns a GlobalRef, that has to be released explicitly. > Probably I get this comment wrong, but I was searching for some code that r It is used in IcebergRowReader::WriteCollectionSlot(), it is freed at the end of the function: env->DeleteLocalRef(collection_scanner); http://gerrit.cloudera.org:8080/#/c/21125/3/be/src/exec/iceberg-metadata/iceberg-row-reader.cc File be/src/exec/iceberg-metadata/iceberg-row-reader.cc: http://gerrit.cloudera.org:8080/#/c/21125/3/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@123 PS3, Line 123: // Skip the unsupported type and set it to NULL > Does this set Binary cols NULL now? It shouldn't, in the backend BINARY fields have TYPE_STRING and a separete boolean variable that determines whether it is actually a STRING or a BINARY. See https://github.com/apache/impala/blob/73171cb7164573349bd53a996a51bb7058b778e0/be/src/runtime/types.h#L216. On the other hand, I don't know where top-level BINARY fields are NULLed for metadata tables. http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc File be/src/exec/iceberg-metadata/iceberg-row-reader.cc: http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@125 PS4, Line 125: VLOG(3) << "Skipping unsupported column type: " << slot_desc->type().type; > I think this 'accessed_value' is allocated one level above in the call chai Done. Added the deallocation to the other caller, WriteArrayItem(), too. http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@212 PS4, Line 212: if constexpr (IS_ARRAY) { > Let's consider moving these DCHECKs into the same IF-ELSE at L228-234. If we do that and this function is erroneously called with for example an INT, the DCHECK that fires will be the one added for "item_tuple_desc != nullptr", not the one about the type. I think type info would help more with debugging than info about 'item_tuple_desc'. Also, this is a more basic precondition of the function. http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@220 PS4, Line 220: const TupleDescriptor* item_tuple_desc = slot_desc->children_tuple_descriptor(); > nit: DCHECK item_tuple_desc is not null? Done http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@236 PS4, Line 236: > name is misleading now that this is not just for arrays. remaining_items? Done http://gerrit.cloudera.org:8080/#/c/21125/4/be/src/exec/iceberg-metadata/iceberg-row-reader.cc@297 PS4, Line 297: jobject value; : RETURN_IF_ERROR(met > Can the key of a map be a struct? Done, added a DCHECK instead. http://gerrit.cloudera.org:8080/#/c/21125/3/testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test File testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test: http://gerrit.cloudera.org:8080/#/c/21125/3/testdata/workloads/functional-query/queries/QueryTest/iceberg-metadata-tables.test@761 PS3, Line 761: ==== > Can't we do a map_col.KEY and map_col.VALUE for maps? Would be nice to have We can't, to access KEY and VALUE we have to cross-unnest the collection, and it is not supported yet. See https://issues.apache.org/jira/browse/IMPALA-12853. -- To view, visit http://gerrit.cloudera.org:8080/21125 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8a8b3a574ca45c893315c3b41b33ce4e0eff865a Gerrit-Change-Number: 21125 Gerrit-PatchSet: 6 Gerrit-Owner: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Noemi Pap-Takacs <npaptak...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Comment-Date: Thu, 28 Mar 2024 15:47:36 +0000 Gerrit-HasComments: Yes
