[ https://issues.apache.org/jira/browse/IMPALA-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16781216#comment-16781216 ]
Vihang Karajgaonkar commented on IMPALA-8266: --------------------------------------------- Adding the review link > Event filtering logic may not filter all the events > --------------------------------------------------- > > Key: IMPALA-8266 > URL: https://issues.apache.org/jira/browse/IMPALA-8266 > Project: IMPALA > Issue Type: Sub-task > Reporter: Vihang Karajgaonkar > Assignee: Vihang Karajgaonkar > Priority: Major > > Here is a the logic to filter out create events from a given batch of events. > {code} > Iterator<MetastoreEvent> it = metastoreEvents.iterator(); > // filter out the create events which has a corresponding drop event > later > int fromIndex = 0; > int numFilteredEvents = 0; > int inputSize = metastoreEvents.size(); > while (it.hasNext()) { > MetastoreEvent current = it.next(); > if (fromIndex < metastoreEvents.size() && current.isRemovedAfter( > metastoreEvents.subList(fromIndex + 1, metastoreEvents.size()))) { > LOG.info(current.debugString("Filtering out this event since the > object is " > + "either removed or renamed later in the event stream")); > it.remove(); > numFilteredEvents++; > } > fromIndex++; > } > {code} > If the event list contains CREATE_DATABASE, CREATE_TABLE, DROP_TABLE, > DROP_DATABASE events, it is possible that we only filter out CREATE_DATABASE > event and not CREATE_TABLE event. This is because the {{fromIndex}} above > gets incremented event iteration of the {{while}} loop and hence when it is > evaluating if there is a inverse event later on in the stream, it starts to > look from DROP_DATABASE event onwards. > The fix is simple, the fromIndex needs to be incremented only when the item > from the list is not removed. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org