[
https://issues.apache.org/jira/browse/IMPALA-12851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824834#comment-17824834
]
ASF subversion and git services commented on IMPALA-12851:
----------------------------------------------------------
Commit b7ddbcad0dd6accb559a3f391a897a8c442d1728 in impala's branch
refs/heads/master from Venu Reddy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=b7ddbcad0 ]
IMPALA-12832: Implicit invalidate metadata on event failures
At present, failure in event processing needs manual invalidate
metadata. This patch implicitly invalidates the table upon failures
in processing of table events with new
'invalidate_metadata_on_event_processing_failure' flag. And a new
'invalidate_global_metadata_on_event_processing_failure' flag is
added to global invalidate metadata automatically when event
processor goes to non-active state.
Note: Also introduced a config
'inject_process_event_failure_event_types' for automated tests to
simulate event processor failures. This config is used to specify what
event types can be intentionally failed. This config should only be
used for testing purpose. Need IMPALA-12851 as a prerequisite
Testing:
- Added end-to-end tests to mimic failures in event processor and verified
that event processor is active
- Added unit test to verify the 'auto_global_invalidate_metadata' config
- Passed FE tests
Co-Authored-by: Sai Hemanth Gantasala <saihema...@cloudera.com>
Change-Id: Ia67fc04c995802d3b6b56f79564bf0954b012c6c
Reviewed-on: http://gerrit.cloudera.org:8080/21065
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Process AllocWriteIdEvent do not add txnId to tableWriteIds mapping in Catalog
> ------------------------------------------------------------------------------
>
> Key: IMPALA-12851
> URL: https://issues.apache.org/jira/browse/IMPALA-12851
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Venugopal Reddy K
> Assignee: Venugopal Reddy K
> Priority: Major
>
> *Description:*
> txnId to tableWriteIds mapping in Catalog(i.e.,
> org.apache.impala.catalog.Catalog#txnToWriteIds_) is not added in the
> following cases:
> # When CREATE_TABLE event is followed by ALLOC_WRITE_ID_EVENT for a table in
> the same batch of MetastoreEventsProcessor.processEvents(), process
> AllocWriteIdEvent do not add txnId to tableWriteIds mapping in catalog. It
> happens, because we get table from catalog in the AllocWriteIdEvent
> constructor. And catalog_.getTable() returns null as the CREATE_TABLE is not
> processed by the time of AllocWriteIdEvent construction.Following events get
> the catalog table in the constructors:-
> ## AllocWriteIdEvent,
> ## ReloadEvent,
> ## CommitCompactionEvent.
> # When the table is not loaded, msTbl_ is null. process AllocWriteIdEvent
> returns without updating txnId to tableWriteIds map in catalog.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
