Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/14307 )
Change subject: IMPALA-7506: support global INVALIDATE METADATA in local catalog mode ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/14307/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/14307/5/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1104 PS5, Line 1104: if (ctx.collectFullUpdates || tbl.getCatalogVersion() <= ctx.toVersion) { > I've tried tracking the min catalog versions in local-catalog mode coordina For your concerns, I think it's ok since we already have the logic to force the table being propogated if it has been skipped more than MAX_NUM_SKIPPED_TOPIC_UPDATES(2) times. Why can't we do the same thing just after global invalidate metadata? If there are too many concurrent DDLs, global invalidate metadata may also hang for a while in the existing implementation. Because the coordinator is waiting for catalogd to propogate the skipped table, and catalogd is trying to propogate it after skips it 2 times but waiting for acquiring the table lock that is hold by the concurrent DDLs. (same as IMPALA-6671) -- To view, visit http://gerrit.cloudera.org:8080/14307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib61a7ab1ffa062620ffbc2dadc34bd7a8ca9e549 Gerrit-Change-Number: 14307 Gerrit-PatchSet: 5 Gerrit-Owner: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@apache.org> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Todd Lipcon <t...@apache.org> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Comment-Date: Wed, 16 Oct 2019 02:41:21 +0000 Gerrit-HasComments: Yes
