[
https://issues.apache.org/jira/browse/IMPALA-13293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17874059#comment-17874059
]
ASF subversion and git services commented on IMPALA-13293:
----------------------------------------------------------
Commit 6323455425f946eee5c9bd78a94b99bf422c51de in impala's branch
refs/heads/master from Csaba Ringhofer
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=632345542 ]
IMPALA-13293: Fix too long wait for initial catalog update
Coordinators always waited 2s in Frontend.WaitForCatalog() for the
first catalog update to arrive, even if the update arrived quicker.
This was obvious in case of LocalCatalog, but ImpaladCatalog looked
as if it would wake up when the update arrives by notifying
catalogUpdateEventNotifier_. In reality the arrival of the catalog
update led FeCatalogManager to create a new ImpaladCatalog instance
while Frontend was still waiting for the old instance which never
got notified.
The change fixes the wait/notify mechanism in ImpaladCatalog and
implements a similar logic for LocalCatalog in CatalogdMetaProvider.
Makes starting impala cluster faster:
ImpaladCatalog: ~12s->~11s
LocalCatalog: ~11->~10s
Change-Id: I4da1d0c0d25f80b02c39be863431201d1b913c5d
Reviewed-on: http://gerrit.cloudera.org:8080/21655
Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
> LocalCatalog's waitForCatalogUpdate() sleeps too much
> -----------------------------------------------------
>
> Key: IMPALA-13293
> URL: https://issues.apache.org/jira/browse/IMPALA-13293
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Frontend
> Reporter: Csaba Ringhofer
> Priority: Major
>
> Unlike ImpaladCatalog's waitForCatalogUpdate() the LocalCatalog version
> doesn't use a conditional variable and simply waits for timeoutMs. The
> timeout comes from MAX_CATALOG_UPDATE_WAIT_TIME_MS, which is 2 seconds. This
> means the the function will wait 2 seconds even it the catalog update arrived
> in the meantime. This 2 seconds is often nearly completely added to the
> impala cluster startup time.
> The sleep was added in
> https://gerrit.cloudera.org/#/c/11472/3/fe/src/main/java/org/apache/impala/catalog/local/LocalCatalog.java
> Update: realized that this also doesn't work well for ImpaladCatalog - the
> FeCatalogManager creates a new ImpaladCatalog when a full topic update
> arrives, so the Object that waitForCatalogUpdate() waits for is never
> notified. My impression is that was broken a long time ago, even before
> LocalCatalog was added.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org
