[ https://issues.apache.org/jira/browse/IMPALA-8968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16937266#comment-16937266 ]
Quanlong Huang commented on IMPALA-8968: ---------------------------------------- [~anuragmantri], I was unlucky to reproduce the bug... Did I miss anything? {code:java} $ bin/start-impala-cluster.py --catalogd_args="--hms_event_polling_interval_s=2" $ bin/impala-shell.sh [localhost:21000] default> CREATE DATABASE test; [localhost:21000] default> CREATE FUNCTION test.fn_test (INT, STRING) RETURNS BOOLEAN LOCATION '/test-warehouse/impala-hive-udfs.jar' SYMBOL="org.apache.impala.TestUdf"; [localhost:21000] default> DROP FUNCTION test.fn_test(INT, STRING); [localhost:21000] default> DROP DATABASE test CASCADE; {code} The only difference is that I use '/test-warehouse/impala-hive-udfs.jar' and SYMBOL="org.apache.impala.TestUdf" since I can't find '/test-warehouse/dummy.jar' in my minicluster. Can't find the error in catalogd. Is it a permanent issue or should I try more times to let it emerge? > Fix self-event detection on database events. > -------------------------------------------- > > Key: IMPALA-8968 > URL: https://issues.apache.org/jira/browse/IMPALA-8968 > Project: IMPALA > Issue Type: Bug > Components: Frontend > Reporter: Anurag Mantripragada > Assignee: Anurag Mantripragada > Priority: Major > > When event processing is turned on, self-events are not detected for DATABASE > level events (create/alter/drop database). Reproduced using the below > statements: > {code:java} > CREATE DATABASE test; > CREATE FUNCTION test.fn_test (INT, STRING) RETURNS BOOLEAN LOCATION > '/test-warehouse/dummy.jar' SYMBOL='com.cloudera.impala.TestUdf'; > DROP FUNCTION test.fn_test(INT, STRING); > DROP DATABASE test CASCADE; {code} > Since the events processor could not detect self-events, it will try to > process the ALTER_DATABASE event created by dropping the function. However, > it doesn't find the DB and errors out like below: > {code:java} > I0923 16:09:46.042317 6727 MetastoreEventsProcessor.java:480] Received 2 > events. Start event id : 30077 > I0923 16:09:46.042317 6727 MetastoreEventsProcessor.java:480] Received 2 > events. Start event id : 30077 > I0923 16:09:46.042853 6727 MetastoreEvents.java:382] EventId: 30078 > EventType: ALTER_DATABASE Creating event 30078 of type ALTER_DATABASE on > database test > I0923 16:09:46.044441 6727 MetastoreEvents.java:382] EventId: 30079 > EventType: DROP_DATABASE Creating event 30079 of type DROP_DATABASE on > database test > I0923 16:09:46.050657 6727 MetastoreEvents.java:231] Total number of events > received: 2 Total number of events filtered out: 0 > I0923 16:09:46.051167 6727 MetastoreEvents.java:382] EventId: 30078 > EventType: ALTER_DATABASE Received exception Database test not found. > Ignoring self-event evaluation > E0923 16:09:46.056273 6727 MetastoreEventsProcessor.java:525] Unexpected > exception received while processing event > Java exception > follows:org.apache.impala.catalog.events.MetastoreNotificationException: > Unable to process event 30078 of type ALTER_DATABASE. Event processing will > be stopped. > at > org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:614) > at > org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:511) > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > Caused by: org.apache.impala.catalog.DatabaseNotFoundException: Database test > not found > at > org.apache.impala.catalog.CatalogServiceCatalog.updateDb(CatalogServiceCatalog.java:1060) > at > org.apache.impala.catalog.events.MetastoreEvents$AlterDatabaseEvent.process(MetastoreEvents.java:1225) > at > org.apache.impala.catalog.events.MetastoreEvents$MetastoreEvent.processIfEnabled(MetastoreEvents.java:316) > at > org.apache.impala.catalog.events.MetastoreEventsProcessor.processEvents(MetastoreEventsProcessor.java:609) > ... 8 more > E0923 16:09:46.056489 6727 MetastoreEventsProcessor.java:631] Notification > event is null > W0923 16:09:56.057376 6727 MetastoreEventsProcessor.java:504] Event > processing is skipped since status is ERROR. Last synced event id is > 30077{code} > > -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org For additional commands, e-mail: issues-all-h...@impala.apache.org