Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22871 )
Change subject: IMPALA-12162: Checksum files before lock in INSERT ...................................................................... Patch Set 8: (2 comments) http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7348 PS7, Line 7348: Map<String, FileMetadata> fileMetadata = getFileMetadata( : feFsTable, update.getUpdated_partitions().values(), catalogTimeline); > From this point on, update.getUpdated_partitions() and fileMetadata is unch That potentially adds more overhead to creating and tearing down the thread pool in ParallelFileMetadataLoader in the next patch. But I came up with something similar. http://gerrit.cloudera.org:8080/#/c/22871/7/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7910 PS7, Line 7910: if (metadata == null) continue; This isn't a good idea because then FilesAdded and FilesAddedChecksum might not line up. -- To view, visit http://gerrit.cloudera.org:8080/22871 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I18f9686f5d53cf1e7c384684c25427fb5353e2af Gerrit-Change-Number: 22871 Gerrit-PatchSet: 8 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 20 May 2025 23:15:26 +0000 Gerrit-HasComments: Yes
