Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/12118 )
Change subject: IMPALA-7970 : Add support for metastore event based automatic invalidate ...................................................................... Patch Set 19: (1 comment) Thanks for addressing my comments. I think there might be one (probably rare) race that is possible. http://gerrit.cloudera.org:8080/#/c/12118/19/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java: http://gerrit.cloudera.org:8080/#/c/12118/19/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java@269 PS19, Line 269: lastSyncedEventId_ = I think fetching the event ID here can lead to a race where we miss a notification, e.g.: 1. CatalogServiceCatalog.reset() is called 2. CatalogServiceCatalog.reset() loads DB and table metadata, which does not include table X. 3. Table X is added to the HMS, e.g. via Hive 4. This function fetches the last event ID 5. The processor starts processing events, but doesn't see the table X create event. I think the solution is to fetch the last event ID earlier in CatalogServiceCatalog.reset() and pass it into this function. -- To view, visit http://gerrit.cloudera.org:8080/12118 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic70b27780560b7ac9b33418d132b36cd0ca4abf7 Gerrit-Change-Number: 12118 Gerrit-PatchSet: 19 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Paul Rogers <prog...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Comment-Date: Thu, 17 Jan 2019 22:22:34 +0000 Gerrit-HasComments: Yes
