[
https://issues.apache.org/jira/browse/IMPALA-12468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17822520#comment-17822520
]
Manish Maheshwari commented on IMPALA-12468:
--------------------------------------------
Would it not be better that we ensure that EventProcessor never goes into
NEED_INVALIDATE state, For any exception, either it should catch it or log it
and continue with the next event so that an admin does not need to check if EP
is stopped and then manually start it again. Perhaps we should also make sure
that EP stopped is communicated to all Impala coordiantors and we stop
ingesting queries as they have a high risk of returning incorrect results.
> Add the ability to update EventProcessorStatus
> ----------------------------------------------
>
> Key: IMPALA-12468
> URL: https://issues.apache.org/jira/browse/IMPALA-12468
> Project: IMPALA
> Issue Type: Improvement
> Components: be, Catalog, fe
> Reporter: Maxwell Guo
> Assignee: Maxwell Guo
> Priority: Minor
>
> Once the impala and hive's status is missmatched , and the
> EventProcessorStatus become NEED_INVALIDATEļ¼ we usually use invalidate
> metadata to reset the catalog instance. And then impala will update the
> status to ACTIVE .
> But if impala contains many tables , the cost of invalidate is a bit high for
> a global invalidate. So we may invalidate metadata for tables one by one for
> these incremental changed table. For example , we have 1000,000,000,000
> tables but only some of the table event process occurs CatalogException and
> MetastoreNotificationNeedsInvalidateException was thrown. I think there is no
> need to invalidate all table caches in order to reset the catalog instance
> see [here
> |https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2088].
>
> MetaStoresProcessor 's async update process will not update the currentStatus
> when the status is not ACTIVE, see
> [here|https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/events/MetastoreEventsProcessor.java#L876]
> So what about add a new SQL grammar : SET EVENT STATUS ${status} ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
