Sourabh Goyal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17697 )
Change subject: IMPALA-10801: Check the latest compaction Id before serving ACID table ...................................................................... Patch Set 11: (1 comment) http://gerrit.cloudera.org:8080/#/c/17697/10/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/17697/10/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@2131 PS10, Line 2131: partsToBeRefreshed = @Yu-Wen: If there are multiple getOrLoad requests that end up at line 2131 for the same table, each one of them would make HMS api call to detect the partitions for refreshing metadata. It is not efficient because we are making repeated HMS api calls specially if the partitioned table is large. If feel the later requests for the same table could piggyback on the first request (just like loadSync does). The other potential issue I see is with refreshFileMetada method. By the time this method is called, version read lock is released. What if there is another getOrLoad request that replaces the whole table around the same time when we are inside refreshFileMetadata()? Though it should not cause any issue with the cache consistency but we are unnecessarily updating the table that has already been replaced. -- To view, visit http://gerrit.cloudera.org:8080/17697 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I86a112a77980fef7f6238978bc9668a65262101e Gerrit-Change-Number: 17697 Gerrit-PatchSet: 11 Gerrit-Owner: Yu-Wen Lai <yu-wen....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Sourabh Goyal <soura...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Yu-Wen Lai <yu-wen....@cloudera.com> Gerrit-Comment-Date: Thu, 29 Jul 2021 17:33:44 +0000 Gerrit-HasComments: Yes
