[jira] [Updated] (IMPALA-12771) Impala catalogd events-skipped may mark the wrong number
[ https://issues.apache.org/jira/browse/IMPALA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated IMPALA-12771: - Fix Version/s: Impala 4.5.0 (was: Impala 4.4.2) > Impala catalogd events-skipped may mark the wrong number > > > Key: IMPALA-12771 > URL: https://issues.apache.org/jira/browse/IMPALA-12771 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > Fix For: Impala 4.5.0 > > > See the description of [event-skipped > metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] > > {code:java} > // total number of events which are skipped because of the flag setting or > // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were > ignored > // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in > the catalogd. > {code} > > As for CREATE and DROP event on Database/Table/Partition (Also AddPartition > is inclued) when we found that the table/database when the database or table > is not found in the cache then we will skip the event process and make the > event-skipped metric +1. > But I found that there is some question here for alter table and Reload event: > * For Reload event that is not describe in the description of events-skipped, > but the value is +1 when is oldevent; > * Besides if the table is in blacklist the metric will also +1 > In summary, I think this description is inconsistent with the actual > implementation. > So can we also mark the events-skipped metric for alter partition events and > modify the > description to be all the events skipped -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12771) Impala catalogd events-skipped may mark the wrong number
[ https://issues.apache.org/jira/browse/IMPALA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated IMPALA-12771: - Fix Version/s: Impala 4.4.2 > Impala catalogd events-skipped may mark the wrong number > > > Key: IMPALA-12771 > URL: https://issues.apache.org/jira/browse/IMPALA-12771 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > Fix For: Impala 4.4.2 > > > See the description of [event-skipped > metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] > > {code:java} > // total number of events which are skipped because of the flag setting or > // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were > ignored > // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in > the catalogd. > {code} > > As for CREATE and DROP event on Database/Table/Partition (Also AddPartition > is inclued) when we found that the table/database when the database or table > is not found in the cache then we will skip the event process and make the > event-skipped metric +1. > But I found that there is some question here for alter table and Reload event: > * For Reload event that is not describe in the description of events-skipped, > but the value is +1 when is oldevent; > * Besides if the table is in blacklist the metric will also +1 > In summary, I think this description is inconsistent with the actual > implementation. > So can we also mark the events-skipped metric for alter partition events and > modify the > description to be all the events skipped -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12771) Impala catalogd events-skipped may mark the wrong number
[ https://issues.apache.org/jira/browse/IMPALA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanlong Huang updated IMPALA-12771: Epic Link: IMPALA-11531 > Impala catalogd events-skipped may mark the wrong number > > > Key: IMPALA-12771 > URL: https://issues.apache.org/jira/browse/IMPALA-12771 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > > See the description of [event-skipped > metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] > > {code:java} > // total number of events which are skipped because of the flag setting or > // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were > ignored > // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in > the catalogd. > {code} > > As for CREATE and DROP event on Database/Table/Partition (Also AddPartition > is inclued) when we found that the table/database when the database or table > is not found in the cache then we will skip the event process and make the > event-skipped metric +1. > But I found that there is some question here for alter table and Reload event: > * For Reload event that is not describe in the description of events-skipped, > but the value is +1 when is oldevent; > * Besides if the table is in blacklist the metric will also +1 > In summary, I think this description is inconsistent with the actual > implementation. > So can we also mark the events-skipped metric for alter partition events and > modify the > description to be all the events skipped -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org
[jira] [Updated] (IMPALA-12771) Impala catalogd events-skipped may mark the wrong number
[ https://issues.apache.org/jira/browse/IMPALA-12771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated IMPALA-12771: - Description: See the description of [event-skipped metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] {code:java} // total number of events which are skipped because of the flag setting or // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were ignored // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in the catalogd. {code} As for CREATE and DROP event on Database/Table/Partition (Also AddPartition is inclued) when we found that the table/database when the database or table is not found in the cache then we will skip the event process and make the event-skipped metric +1. But I found that there is some question here for alter table and Reload event: * For Reload event that is not describe in the description of events-skipped, but the value is +1 when is oldevent; * Besides if the table is in blacklist the metric will also +1 In summary, I think this description is inconsistent with the actual implementation. So can we also mark the events-skipped metric for alter partition events and modify the description to be all the events skipped was: See the description of [event-skipped metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] {code:java} // total number of events which are skipped because of the flag setting or // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were ignored // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in the catalogd. {code} As for CREATE and DROP event on Database/Table/Partition (Also AddPartition is inclued) when we found that the table/database when the database or table is not found in the cache then we will skip the event process and make the event-skipped metric +1. But I found that there is some question here for alter table and Reload event: * For alter table if renaming a table , the events-skipped metric will also +1 ,see [oldTblRemoved to be false |https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEvents.java#L1653] * For Reload event that is not describe in the description of events-skipped, but the value is +1 when is oldevent; * Besides if the table is in blacklist the metric will also +1 In summary, I think this description is inconsistent with the actual implementation. So can we also mark the events-skipped metric for alter partition events and modify the description to be all the events skipped > Impala catalogd events-skipped may mark the wrong number > > > Key: IMPALA-12771 > URL: https://issues.apache.org/jira/browse/IMPALA-12771 > Project: IMPALA > Issue Type: Bug > Components: Catalog >Reporter: Maxwell Guo >Assignee: Maxwell Guo >Priority: Minor > > See the description of [event-skipped > metric|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L237] > > {code:java} > // total number of events which are skipped because of the flag setting or > // in case of [CREATE|DROP] events on [DATABASE|TABLE|PARTITION] which were > ignored > // because the [DATABASE|TABLE|PARTITION] was already [PRESENT|ABSENT] in > the catalogd. > {code} > > As for CREATE and DROP event on Database/Table/Partition (Also AddPartition > is inclued) when we found that the table/database when the database or table > is not found in the cache then we will skip the event process and make the > event-skipped metric +1. > But I found that there is some question here for alter table and Reload event: > * For Reload event that is not describe in the description of events-skipped, > but the value is +1 when is oldevent; > * Besides if the table is in blacklist the metric will also +1 > In summary, I think this description is inconsistent with the actual > implementation. > So can we also mark the events-skipped metric for alter partition events and > modify the > description to be all the events skipped -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org