Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/12591 )
Change subject: IMPALA-7972 Detect self-events to avoid unnecessary invalidates ...................................................................... Patch Set 4: (4 comments) http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java File fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java: http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@738 PS4, Line 738: public Collection<Long> getInFlightVersionsForEvents(String dbName, String tblName) > Should be a list: evens are ordered and must be ignored in the order that t Done http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java@765 PS4, Line 765: public void removeFromInFlightVersionsForEvents(String dbName, String tblName, > This form introduces a race condition. Can you do something like: Discussed this with Paul and agreed that this race condition may not be a problem as of now, since any other thread which can possible modify this list will either add a new version or invalidate the table. In both the cases, the code works as expected. I have added a TODO in MetastoreEvents class to improve this as a followup item later. http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@889 PS4, Line 889: protected Collection<Long> pendingVersionNumbersFromCatalog_ = Collections.EMPTY_LIST; > Again, this introduces a race condition if some other thread changes the ta Will be done as a followup since it is not critical as of now. This is related to invalidateOrIgnore(versionNumber) approach which was suggested. http://gerrit.cloudera.org:8080/#/c/12591/4/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@919 PS4, Line 919: protected boolean isSelfEvent() throws CatalogException { > I see. You want to identify outside of a lock if this is a self event. The Thanks for suggestion, As discussed this can be done as a followup in a separate patch. -- To view, visit http://gerrit.cloudera.org:8080/12591 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I6db0d7f7fe465158fc8cb9d6b6b57a321827b353 Gerrit-Change-Number: 12591 Gerrit-PatchSet: 4 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Bharath Krishna <bhar...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Paul Rogers <prog...@cloudera.com> Gerrit-Reviewer: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Comment-Date: Thu, 28 Feb 2019 00:49:21 +0000 Gerrit-HasComments: Yes
