Sourabh Goyal has posted comments on this change. ( http://gerrit.cloudera.org:8080/17848 )
Change subject: IMPALA-9857: Batching of consecutive partition events ...................................................................... Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/17848/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java File fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java: http://gerrit.cloudera.org:8080/#/c/17848/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1641 PS2, Line 1641: return versionNumberFromEvent_ == versionNumberOfEvent && serviceIdFromEvent_.equals( Would it not be more performant if we batch alter partition events for the same table from other clusters as well? While processing a batched event, we are anyways looping over events to check if they can be skipped. Can add one more check for self events and ignore those. The BatchPartitionEvent process() code where we are looping over. for (T event : batchedEvents_) { if (event.canBeSkipped()) { ignoredPartitions.add(event); } } http://gerrit.cloudera.org:8080/#/c/17848/2/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java@1728 PS2, Line 1728: this.baseEvent_ = baseEvent; Should we assert here that baseEvent can only be either AlterPartitionEvent or InsertEvent? -- To view, visit http://gerrit.cloudera.org:8080/17848 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5d27a68a64436d31731e9a219b1efd6fc842de73 Gerrit-Change-Number: 17848 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar <vih...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Quanlong Huang <huangquanl...@gmail.com> Gerrit-Reviewer: Sourabh Goyal <soura...@cloudera.com> Gerrit-Comment-Date: Thu, 16 Sep 2021 17:56:13 +0000 Gerrit-HasComments: Yes
