k.venureddy2...@gmail.com has posted comments on this change. ( http://gerrit.cloudera.org:8080/20367 )
Change subject: IMPALA-10976: Sync db/table in catalogD to latest HMS event id for all DDLS from Impala clients ...................................................................... Patch Set 2: (4 comments) few observations http://gerrit.cloudera.org:8080/#/c/20367/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/20367/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1006 PS2, Line 1006: msTbl How do we deal with the case when table is renamed through external source(hive/other catalogd) and the rename event is still not synced on this catalogd ? http://gerrit.cloudera.org:8080/#/c/20367/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1007 PS2, Line 1007: tbl = catalog_.updateMetastoreTable(msTbl, tbl); We don't seem to reload table ? Probably table schema, partitions would have changed. http://gerrit.cloudera.org:8080/#/c/20367/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1011 PS2, Line 1011: long oldCatalogVersion = tbl.getCatalogVersion(); I believe, we need to get the oldCatalogVersion before updating tbl. Because addTableToCatalogUpdate() is done only when oldCatalogVersion is different from tbl.getCatalogVersion(). http://gerrit.cloudera.org:8080/#/c/20367/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2398 PS2, Line 2398: long newCatalogVersion = catalog_.incrementAndGetCatalogVersion(); It is good to get the newCatalogVersion after catalog_.updateMetastoreTable(). Otherwise newCatalogVersion would be smaller than table.getCatalogVersion(). -- To view, visit http://gerrit.cloudera.org:8080/20367 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia250d0a943838086c187e5cb7c60035e5a564bbf Gerrit-Change-Number: 20367 Gerrit-PatchSet: 2 Gerrit-Owner: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Reviewer: Anonymous Coward <k.venureddy2...@gmail.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sai Hemanth Gantasala <saihema...@cloudera.com> Gerrit-Comment-Date: Fri, 29 Sep 2023 20:16:37 +0000 Gerrit-HasComments: Yes
