Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16549 )
Change subject: IMPALA-6671: [WIP] Skip locked tables from topic updates ...................................................................... Patch Set 1: (3 comments) I think if this approach works out it would meaningfully improve metadata for production workloads - seems like it should reduce the number of hiccups that non-long-running-ddl queries experience, although obviously with the approach they can eventually get stuck behind a DDL. We'd expect that potentially, this could have some impact on tables where there are multiple long-running DDLs, right? E.g. that table might skip topic updates and delay queries waiting for that particular table. I think that seems fine, but just wanted to be sure. http://gerrit.cloudera.org:8080/#/c/16549/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16549/1//COMMIT_MSG@9 PS1, Line 9: This change adds a mechanism for topic-update thread question, that maybe will be answered as I read the patch - if an update is skipped, does the topic update collection thread get woken up when the lock is released? http://gerrit.cloudera.org:8080/#/c/16549/1//COMMIT_MSG@23 PS1, Line 23: This solution unblocks topic-updates for a while until a Maybe this is the best we can do at the moment, but it does seem to have similar problems to the IMPALA-5058 solution where it does help with latency when it's blocked for shorter periods of time but still gets blocked when a lock is held for a long time. http://gerrit.cloudera.org:8080/#/c/16549/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/16549/1/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@1392 PS1, Line 1392: if (tblVersion > ctx.toVersion && I've seen some cases in production workloads where the topic update thread gets stuck acquiring the table lock, but then has to skip the update anywhere here. Is there any way we can move this check outside of the lock acquisition so that it can exit early in that case? -- To view, visit http://gerrit.cloudera.org:8080/16549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic657b96edbcdc94c6b906e7ca59291f4e4715655 Gerrit-Change-Number: 16549 Gerrit-PatchSet: 1 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Shant Hovsepian <sh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Comment-Date: Tue, 06 Oct 2020 19:50:09 +0000 Gerrit-HasComments: Yes
