Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/22872 )
Change subject: IMPALA-12162: Use thread pool to collect checksums ...................................................................... Patch Set 16: (4 comments) http://gerrit.cloudera.org:8080/#/c/22872/15/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java File fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java: http://gerrit.cloudera.org:8080/#/c/22872/15/fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java@131 PS15, Line 131: /** : * Store the loaded FDs from loader into th > nit: A method documentation would be nice to have here. Done http://gerrit.cloudera.org:8080/#/c/22872/15/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/22872/15/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7876 PS15, Line 7876: > Should this be "final String"? It passed down to lambda expression below. It qualifies as "effectively final" for the purpose of the lambda, so I didn't overdo variable decoration. http://gerrit.cloudera.org:8080/#/c/22872/15/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7883 PS15, Line 7883: private Map<String, List<FileMetadata>> getFileMetadata(FeFsTable table, : Map<String, TUpdatedPartition> updatedPartitions, String debugAction, : EventSequence catalogTimeline) { : if (!shouldGenerateInsertEvents(table)) return null; : boolean isPartitioned = table.isPartitioned(); : boolean isTransactional = AcidUtils.isTransactionalTable(table); : : // Get table file system with table location. : final FileSystem tableFs = getFileSystemOrNull(ta > Why this need to use Supplier if only used locally? I couldn't figure out how to directly invoke the lambda; using Optional.<>default.orElseGet was another workaround someone suggested for exception handling while initializing a final variable. I pulled this into a help method that logs the exception and returns null. http://gerrit.cloudera.org:8080/#/c/22872/15/tests/query_test/test_insert.py File tests/query_test/test_insert.py: http://gerrit.cloudera.org:8080/#/c/22872/15/tests/query_test/test_insert.py@199 PS15, Line 199: : : class TestInsertQueriesWithDefaultFormat(TestInsertBase): > This can be flaky in slow build flavor (ASAN, TSAN, UBSAN). Ok, the num-loading-threads check is probably sufficient. -- To view, visit http://gerrit.cloudera.org:8080/22872 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I314621104e4757620c0a90d41dd6875bf8855b51 Gerrit-Change-Number: 22872 Gerrit-PatchSet: 16 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Thu, 22 May 2025 20:31:02 +0000 Gerrit-HasComments: Yes
