Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/18043 )
Change subject: IMPALA-11032: Automatic Refresh of Metadata for Local Catalog after Compaction ...................................................................... Patch Set 3: (5 comments) http://gerrit.cloudera.org:8080/#/c/18043/3/be/src/service/impala-server.cc File be/src/service/impala-server.cc: http://gerrit.cloudera.org:8080/#/c/18043/3/be/src/service/impala-server.cc@348 PS3, Line 348: conduct nit, conducted http://gerrit.cloudera.org:8080/#/c/18043/3/be/src/service/impala-server.cc@349 PS3, Line 349: . move to previous line? http://gerrit.cloudera.org:8080/#/c/18043/3/be/src/service/impala-server.cc@349 PS3, Line 349: sends one " : "getLatestCompactionInfo request per table to HMS. suggest you to change it to more generic since end users may not understand these details. Something like "... because Impala makes additional RPCs to hive metastore for each table in a query during the query compilation" http://gerrit.cloudera.org:8080/#/c/18043/3/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/18043/3/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java@530 PS3, Line 530: checkLatestCompaction Logging the time taken for this method will be helpful. http://gerrit.cloudera.org:8080/#/c/18043/3/fe/src/main/java/org/apache/impala/service/Frontend.java File fe/src/main/java/org/apache/impala/service/Frontend.java: http://gerrit.cloudera.org:8080/#/c/18043/3/fe/src/main/java/org/apache/impala/service/Frontend.java@1755 PS3, Line 1755: if (BackendConfig.INSTANCE.isAutoCheckCompaction() && analysisResult.isQueryStmt() : && !queryCtx.isSetTransaction_id()) { : long txnId = openTransaction(queryCtx); : timeline.markEvent("Transaction opened (" + String.valueOf(txnId) + ")"); : } If we open a transaction here then we must make sure that we also commit it when the query completes. I suggest you do it in a followup since I don't think the backend calls back into frontend when the query completes and it is not as straight-forward. -- To view, visit http://gerrit.cloudera.org:8080/18043 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I173ea848917b6a41139b25b80677111463bfdc4b Gerrit-Change-Number: 18043 Gerrit-PatchSet: 3 Gerrit-Owner: Yu-Wen Lai <yu-wen....@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sourabh Goyal <soura...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Yu-Wen Lai <yu-wen....@cloudera.com> Gerrit-Comment-Date: Mon, 29 Nov 2021 23:48:13 +0000 Gerrit-HasComments: Yes
