Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/17253 )
Change subject: IMPALA-6671: Change wait for sync ddl timeout ...................................................................... Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/17253/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/17253/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@3094 PS4, Line 3094: long maxNumAttempts = 5; > Unfortunately, I don't think this approach would work. maxSkippedUpdatesLoc Ah, I see. After thinking this more clear, I think my idea should be changing 'maxNumAttempts' from 5 to 5 * (maxSkippedUpdatesLockContention_ + 1). Please verify my understanding. Let's say there are 6 concurrent DDLs (ddl1, ddl2, ..., ddl6) on table 'foo' and both with sync_ddl=true. And let's say they finishes in this order: t0: ddl1 done, go into waitForSyncDdlVersion() t1-a: catalog topic update thread start collecting updates t1-b: ddl2 done, bump the table version, go into waitForSyncDdlVersion() t1-c: catalog topic update thread finish collecting updates and skip the table since its version is out of scope t2-a: catalog topic update thread start t2-b: ddl3 done, bump the table version, go into waitForSyncDdlVersion() t2-c: catalog topic update thread finish collecting updates and skip the table since its version is out of scope ... t5-a: catalog topic update thread start t5-b: ddl6 done, bump the table version, go into waitForSyncDdlVersion() t5-c: catalog topic update thread finish collecting updates and skip the table since its version is out of scope t6: waitForSyncDdlVersion() for ddl1 returns since 'maxNumAttempts' is reached. This is the expected behavior before IMPALA-6671. Now when skipping locked table is enabled, more rounds of topic updates could be sent in the time range between t1-a and t1-b. So 'maxNumAttempts' is reached quickly if it's still 5. There are at most maxSkippedUpdatesLockContention_ updates can be sent between t1-a and t1-b, the same for t2, t3, ..., t5. So I'm thinking setting maxNumAttempts to 5 * (maxSkippedUpdatesLockContention_ + 1) can achieve the pre-IMPALA-6671 purpose. -- To view, visit http://gerrit.cloudera.org:8080/17253 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I79e64cdec0e6aa7b597a47851b4b5c5441ca5528 Gerrit-Change-Number: 17253 Gerrit-PatchSet: 5 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Comment-Date: Tue, 06 Apr 2021 02:24:57 +0000 Gerrit-HasComments: Yes
