[
https://issues.apache.org/jira/browse/IMPALA-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhi tang updated IMPALA-11731:
------------------------------
Description:
If catalog_topic_mode is minimal, for table updates, catalogd only propagates
the database name, table name and catalog version associated with the table.
After the update thread acquire the table lock, the
catalog-update-gathering-thread can notify the coordinator that the table
metadata has changed, rather than waiting for the table lock to be acquired.
We do not expect the catalog-update-gathering-thread to be blocked because it
will block other unrelated updates, such as the problem described in
IMPALA-6671. It is necessary to deal with this problem in a more effective way.
was:
In the minimal mode of the catalog server, only partition related information
needs to be protected by table locks when processing table updates. So I'm
thinking whether we can avoid the blocking of the table lock on the
catalog-update-gathering-thread in this way:
1. When an update arrives, we first determine which partitions will be affected
by the update after acquiring the table lock;
2. Then the catalog-update-gathering-thread collects the change information of
the partitions. Because we already know the partitions that will be changed, we
do not need to hold table locks;
We do not expect the catalog-update-gathering-thread to be blocked because it
will block other unrelated updates, such as the problem described in
IMPALA-6671. It is necessary to deal with this problem in a more effective way.
> Consider avoiding table locks from blocking the
> catalog-update-gathering-thread when the catalog server's catalog_topic_mode
> is minimal.
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11731
> URL: https://issues.apache.org/jira/browse/IMPALA-11731
> Project: IMPALA
> Issue Type: Improvement
> Reporter: zhi tang
> Priority: Major
>
> If catalog_topic_mode is minimal, for table updates, catalogd only propagates
> the database name, table name and catalog version associated with the table.
> After the update thread acquire the table lock, the
> catalog-update-gathering-thread can notify the coordinator that the table
> metadata has changed, rather than waiting for the table lock to be acquired.
> We do not expect the catalog-update-gathering-thread to be blocked because it
> will block other unrelated updates, such as the problem described in
> IMPALA-6671. It is necessary to deal with this problem in a more effective
> way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
