[
https://issues.apache.org/jira/browse/IMPALA-14967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098585#comment-18098585
]
Joe McDonnell commented on IMPALA-14967:
----------------------------------------
Saw this again on an exhaustive job. Several tests fail during the end-to-end
parallel tests.
{noformat}
E20260721 05:39:41.380749 3275089 MetastoreEvents.java:980] EventId: 58870
EventType: DROP_TABLE Target: test_convert_table_36ca0746.special_chars. Failed
to handle event processing failure for table:
test_convert_table_36ca0746.special_chars
Java exception follows:
java.lang.NullPointerException: Cannot invoke
"org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl"
is null
at
org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
at
org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
at
org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
at
org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
at
org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
at
org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
E20260721 05:39:41.382032 3275089 TableEventExecutor.java:502] Exception
occurred for executor: DbEventExecutor-1.TableEventExecutor-3
E20260721 05:39:41.382186 3275089 MetastoreEventsProcessor.java:1407]
Unexpected exception received while processing event
Java exception follows:
java.lang.NullPointerException: Cannot invoke
"org.apache.impala.catalog.Table.setCatalogVersion(long)" because "removedTbl"
is null
at
org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
at
org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
at
org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
at
org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
at
org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
at
org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
at
java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at
java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
E20260721 05:39:41.382831 3275089 MetastoreEventsProcessor.java:1783] Event id:
58870
Event Type: DROP_TABLE
Event time: 0
Database name: test_convert_table_36ca0746
Table name: special_chars
Event message: null
... lots of time ...
E20260721 06:17:51.285123 3275924 CatalogServiceCatalog.java:4733]
344d1e32ecef548b:7f1a377400000000] Current state of HMS event processor is
ERROR{noformat}
The test failures are all after this point, and have a variety of symptoms
(tables missing, event processor in error state, etc).
> Event processor hits NullPointerException for
> CatalogOpExecutor.removeTableIfNotAddedLater() and goes into ERROR state
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14967
> URL: https://issues.apache.org/jira/browse/IMPALA-14967
> Project: IMPALA
> Issue Type: Bug
> Reporter: Pranav Yogi Lodha
> Priority: Critical
> Labels: broken-build, flaky
>
> h3. Error Message
> Exception: Event processor is not working. Status: ERROR. Error msg:
> 2026-05-01T06:48:43.942372438 Unexpected exception received while processing
> event java.lang.NullPointerException: Cannot invoke
> "org.apache.impala.catalog.Table.setCatalogVersion(long)" because
> "removedTbl" is null at
> org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
> at
> org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
> at
> org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
> at
> org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
> at
> org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
> at
> org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
> at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:840) Event id: 46381 Event
> Type: DROP_TABLE Event time: 0 Database name:
> test_count_star_optimization_in_complex_query_a19cc8f9 Table name:
> expected_merge_exclude_columns Event message: null
> h3. Stacktrace
> /data/jenkins/workspace/impala-cdh_main_staging-core-ubsan-arm/repos/Impala/tests/metadata/test_testcase_builder.py:38:
> in test_query_without_from self._test_export_and_import(0, 0, 0, "SELECT 5 *
> 20") self = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object
> at 0xffff92224d50>
> /data/jenkins/workspace/impala-cdh_main_staging-core-ubsan-arm/repos/Impala/tests/metadata/test_testcase_builder.py:102:
> in _test_export_and_import
> EventProcessorUtils.wait_for_event_processing(self) hdfs_path =
> '/tmp/impala-testcase-data-d388835f-83ef-4ee3-afd3-34950744b114' index = 23
> num_dbs = 0 num_tbls = 0 num_views = 0 original_plan = ['Max Per-Host
> Resource Reservation: Memory=0B Threads=1', 'Per-Host Resource Estimates:
> Memory=10MB', 'Codegen disabl...ner', 'Analyzed query: SELECT CAST(100 AS
> SMALLINT)', '', 'F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1', ...]
> query = 'SELECT 5 * 20' result =
> <tests.common.impala_connection.ImpylaHS2ResultSet object at 0xffff921c2310>
> self = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object at
> 0xffff92224d50> tbls = () testcase_generate_query = "COPY TESTCASE TO '/tmp'
> SELECT 5 * 20" testcase_path =
> "'hdfs://localhost:20500/tmp/impala-testcase-data-d388835f-83ef-4ee3-afd3-34950744b114'"
> tmp_path = '/tmp' views = () util/event_processor_utils.py:92: in
> wait_for_event_processing
> EventProcessorUtils.wait_for_event_processing_impl(test_suite.hive_client,
> error_status_possible = False impala_cluster =
> <tests.common.impala_cluster.ImpalaCluster object at 0xffff91d948d0>
> test_suite = <tests.metadata.test_testcase_builder.TestTestcaseBuilder object
> at 0xffff92224d50> timeout = 10 util/event_processor_utils.py:106: in
> wait_for_event_processing_impl
> EventProcessorUtils.wait_for_synced_event_id(timeout, error_status_possible,
> current_event_id = 49314 error_status_possible = False hive_client =
> <impala_thrift_gen.hive_metastore.ThriftHiveMetastore.Client object at
> 0xffff91b49310> impala_cluster = <tests.common.impala_cluster.ImpalaCluster
> object at 0xffff91d948d0> timeout = 10 util/event_processor_utils.py:67: in
> wait_for_synced_event_id raise Exception("Event processor is not working.
> Status: \{0}. Error msg: \{1}" E Exception: Event processor is not working.
> Status: ERROR. Error msg: 2026-05-01T06:48:43.942372438 E Unexpected
> exception received while processing event E java.lang.NullPointerException:
> Cannot invoke "org.apache.impala.catalog.Table.setCatalogVersion(long)"
> because "removedTbl" is null E at
> org.apache.impala.service.CatalogOpExecutor.removeTableIfNotAddedLater(CatalogOpExecutor.java:874)
> E at
> org.apache.impala.catalog.events.RenameTableBarrierEvent.process(RenameTableBarrierEvent.java:209)
> E at
> org.apache.impala.catalog.events.RenameTableBarrierEvent.processIfEnabled(RenameTableBarrierEvent.java:192)
> E at
> org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.processEvent(TableEventExecutor.java:320)
> E at
> org.apache.impala.catalog.events.TableEventExecutor$TableProcessor.process(TableEventExecutor.java:347)
> E at
> org.apache.impala.catalog.events.TableEventExecutor.process(TableEventExecutor.java:499)
> E at
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
> E at
> java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) E
> at
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
> E at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> E at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> E at java.base/java.lang.Thread.run(Thread.java:840) E E Event id: 46381 E
> Event Type: DROP_TABLE E Event time: 0 E Database name:
> test_count_star_optimization_in_complex_query_a19cc8f9 E Table name:
> expected_merge_exclude_columns E Event message: null
> TIMEOUT_MULTIPLIER_IF_THERE_IS_PROGRESS = 10 current_synced_id = 46380
> end_time = 1777644392.8926485 error_msg =
> '2026-05-01T06:48:43.942372438\nUnexpected exception received while
> processing event\njava.lang.NullPointerException:
> ...est_count_star_optimization_in_complex_query_a19cc8f9\nTable name:
> expected_merge_exclude_columns\nEvent message: null' error_status_possible =
> False last_synced_id = 46380 last_synced_time = 1777644292.89848 status =
> 'ERROR' status_list = ['ACTIVE', 'PAUSED'] t = 1777644292.8984802
> target_event_id = 49314 timeout = 10 total_timeot = 100
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
