[
https://issues.apache.org/jira/browse/IMPALA-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098811#comment-18098811
]
Quanlong Huang commented on IMPALA-14618:
-----------------------------------------
[~VenuReddy] has a patch that is under review for a while:
[https://gerrit.cloudera.org/c/23823/]
I think the solution fixes the symptom by not allowing catalog resets run
concurrently with any ongoing event processing. This is true when hierarchical
event processing is disabled so we don't have this issue before. Now
hierarchical event processing is enabled by default, more events could be
processed in parallel so we have a higher possibility to run reset concurrently
when a slow event processing is ongoing, which is a performance regression for
reset.
I proposed another solution that aims to fix the root cause - the race between
catalog reset and event processing: [https://gerrit.cloudera.org/c/24629]
> TestCatalogdHA.test_failover_catchup_timeout_and_reset failing
> --------------------------------------------------------------
>
> Key: IMPALA-14618
> URL: https://issues.apache.org/jira/browse/IMPALA-14618
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 5.0.0
> Reporter: Surya Hebbar
> Assignee: Venugopal Reddy K
> Priority: Blocker
> Labels: broken-build, flaky
> Attachments:
> catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-1ce4.vpc.cloudera.com.jenkins.log.INFO.20251207-114532.2336954,
>
> catalogd.impala-ec2-redhat86-m6i-4xlarge-ondemand-1ce4.vpc.cloudera.com.jenkins.log.INFO.20251207-114532.2336965,
>
> impalad.impala-ec2-redhat86-m6i-4xlarge-ondemand-1ce4.vpc.cloudera.com.jenkins.log.INFO.20251207-114532.2337035
>
>
> h3. Error Message
> {code}
> assert '26000' in "Query 294e9bcf7bfb219a:72be327b00000000
> failed:\nAnalysisException: Could not resolve path:
> 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n" + where '26000'
> = str(26000) + where 26000 = <tests.common.impala_service.CatalogdService
> object at 0x7f51b29a3520>.service_port + where
> <tests.common.impala_service.CatalogdService object at 0x7f51b29a3520> =
> <CatalogdProcess PID: None
> (/data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/be/build/latest/service/...ist.txt
> -catalog_service_port=26000 -state_store_subscriber_port=23020
> -webserver_port=25020 -enable_catalogd_ha=true)>.service + and "Query
> 294e9bcf7bfb219a:72be327b00000000 failed:\nAnalysisException: Could not
> resolve path: 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n" =
> str(HiveServer2Error("Query 294e9bcf7bfb219a:72be327b00000000
> failed:\nAnalysisException: Could not resolve path:
> 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n"))
> {code}
> h3. Stacktrace
> {code}
> custom_cluster/test_catalogd_ha.py:608: in
> test_failover_catchup_timeout_and_reset
> self._test_metadata_after_failover(
> custom_cluster/test_catalogd_ha.py:787: in _test_metadata_after_failover
> assert str(active_catalogd.service.service_port) in str(e)
> E assert '26000' in "Query 294e9bcf7bfb219a:72be327b00000000
> failed:\nAnalysisException: Could not resolve path:
> 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n"
> E + where '26000' = str(26000)
> E + where 26000 = <tests.common.impala_service.CatalogdService object
> at 0x7f51b29a3520>.service_port
> E + where <tests.common.impala_service.CatalogdService object at
> 0x7f51b29a3520> = <CatalogdProcess PID: None
> (/data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/be/build/latest/service/...ist.txt
> -catalog_service_port=26000 -state_store_subscriber_port=23020
> -webserver_port=25020 -enable_catalogd_ha=true)>.service
> E + and "Query 294e9bcf7bfb219a:72be327b00000000
> failed:\nAnalysisException: Could not resolve path:
> 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n" =
> str(HiveServer2Error("Query 294e9bcf7bfb219a:72be327b00000000
> failed:\nAnalysisException: Could not resolve path:
> 'test_failover_catchup_timeout_and_reset_5d02c505.tbl'\n\n"))
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
