Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/24572 )
Change subject: IMPALA-15171: Null out Iceberg delete file path slot after the join ...................................................................... Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/24572/5//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/24572/5//COMMIT_MSG@9 PS5, Line 9: IcebergScanPlanner materializes the INPUT__FILE__NAME (file path) I have a basic question about design: why do we even use INPUT__FILE__NAME? My point is that we know the list of data files with delete files during planning, and could assign them an int32 id instead (e.g INPUT__FILE__ID), and use it as key instead of INPUT__FILE__NAME. This needs a map of INPUT__FILE__NAME->INPUT__FILE__ID to be applied when reading delet files, but my understanding is that each host knows the input files anyway due to path->host map for directed shuffle. This patch is pretty simple and brings large benefits, so I am not saying to go straight to using ID, but I see that as a longer term solution. It would make the tuple smaller and the lookups faster. Having such ID could be also useful for other things, e.g. mapping back rows to partitions based on source file. http://gerrit.cloudera.org:8080/#/c/24572/5//COMMIT_MSG@10 PS5, Line 10: position-delete join : (IcebergDeleteJoinNode) can use it as a join key. Is the issue still relevant with Iceberg v3, or only v2 tables are affected? -- To view, visit http://gerrit.cloudera.org:8080/24572 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I7e662cb6a98dd3e687185d384731d6be45f91b94 Gerrit-Change-Number: 24572 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Peter Rozsa <[email protected]> Gerrit-Comment-Date: Thu, 30 Jul 2026 09:21:27 +0000 Gerrit-HasComments: Yes
