[
https://issues.apache.org/jira/browse/IMPALA-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-4045.
-----------------------------------
Resolution: Won't Fix
This general problem is solved by the local catalog - IMPALA-7127
> Catalog cache update should not tied to statestore update frequency
> -------------------------------------------------------------------
>
> Key: IMPALA-4045
> URL: https://issues.apache.org/jira/browse/IMPALA-4045
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog, Distributed Exec
> Affects Versions: Impala 2.7.0
> Reporter: Huaisi Xu
> Priority: Major
> Labels: performance, usability
>
> All impalad's catalog cache update depends on statestore's "nature" periodic
> update.
> As a result, impala's {{INVALIDATE METADATA}} and initial "prioritized
> loading" are delayed by up to 2x"statestore_update_frequency_ms" in the same
> connection per coordinator.
> Many use cases have {{statestore_update_frequency_ms}} setting to be larger
> than 20s, so everyone of them are affected by this unnecessary delay.
> (20s-40s)
> Consider make "prioritized loading" universal (not just for catalogd).
> {{REFRESH}} is not affected by this statestore_update_frequency_ms.
> Repro:
> start statestore with higher and higher {{statestore_update_frequency_ms}},
> and you will find initial query(and invalidate metadata) get slower and
> slower...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
