[
https://issues.apache.org/jira/browse/IMPALA-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16941347#comment-16941347
]
ASF subversion and git services commented on IMPALA-8968:
---------------------------------------------------------
Commit 0b68189166cfe27a7234877b5ce52ded6d62e0ab in impala's branch
refs/heads/master from Anurag Mantripragada
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0b68189 ]
IMPALA-8968: Alter database events on dropped database should not put
events processor in error state.
This change is two-fold:
1. If an alter database event is received on database that does not
exist, the event can be safely ignored. The events processor should
only go into an error state if updateDb() fails.
2. This change also adds catalog service identifiers to create/drop
function operations as Impala generates alter database events
with these operations and they should be detected as self-events
and ignored.
Testing:
Add tests to verify both the above changes to
MetastoreEventsProcessorTest.
Change-Id: I5f76136aeff35d1d38fc8b3d9a38da399d36eced
Reviewed-on: http://gerrit.cloudera.org:8080/14296
Reviewed-by: Quanlong Huang <huangquanl...@gmail.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> Fix self-event detection on database events.
> --------------------------------------------
>
> Key: IMPALA-8968
> URL: https://issues.apache.org/jira/browse/IMPALA-8968
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Anurag Mantripragada
> Assignee: Anurag Mantripragada
> Priority: Major
>
> When event processing is turned on, self-events are not detected for DATABASE
> level events (create/alter/drop database). Reproduced using the below
> statements:
> {code:java}
> CREATE DATABASE test;
> CREATE FUNCTION test.fn_test (INT, STRING) RETURNS BOOLEAN LOCATION
> '/test-warehouse/dummy.jar' SYMBOL='com.cloudera.impala.TestUdf';
> DROP FUNCTION test.fn_test(INT, STRING);
> DROP DATABASE test CASCADE; {code}
> Since the events processor could not detect self-events, it will try to
> process the ALTER_DATABASE event created by dropping the function. However,
> it doesn't find the DB and errors out like below:
> {code:java}
> I0923 16:09:46.042317 6727 MetastoreEventsProcessor.java:480] Received 2
> events. Start event id : 30077
> I0923 16:09:46.042317 6727 MetastoreEventsProcessor.java:480] Received 2
> events. Start event id : 30077
> I0923 16:09:46.042853 6727 MetastoreEvents.java:382] EventId: 30078
> EventType: ALTER_DATABASE Creating event 30078 of type ALTER_DATABASE on
> database test
> I0923 16:09:46.044441 6727 MetastoreEvents.java:382] EventId: 30079
> EventType: DROP_DATABASE Creating event 30079 of type DROP_DATABASE on
> database test
> I0923 16:09:46.050657 6727 MetastoreEvents.java:231] Total number of events
> received: 2 Total number of events filtered out: 0
> I0923 16:09:46.051167 6727 MetastoreEvents.java:382] EventId: 30078
> EventType: ALTER_DATABASE Received exception Database test not found.
> Ignoring self-event evaluation
> E0923 16:09:46.056273 6727 MetastoreEventsProcessor.java:525] Unexpected
> exception received while processing event
> Java exception
> follows:org.apache.impala.catalog.events.MetastoreNotificationException:
> Unable to process event 30078 of type ALTER_DATABASE. Event processing will
> be stopped.
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:614)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:511)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.apache.impala.catalog.DatabaseNotFoundException: Database test
> not found
> at
> org.apache.impala.catalog.CatalogServiceCatalog.updateDb(CatalogServiceCatalog.java:1060)
> at
> org.apache.impala.catalog.events.MetastoreEvents$AlterDatabaseEvent.process(MetastoreEvents.java:1225)
> at
> org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:316)
> at
> org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:609)
> ... 8 more
> E0923 16:09:46.056489 6727 MetastoreEventsProcessor.java:631] Notification
> event is null
> W0923 16:09:56.057376 6727 MetastoreEventsProcessor.java:504] Event
> processing is skipped since status is ERROR. Last synced event id is
> 30077{code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
