[
https://issues.apache.org/jira/browse/IMPALA-11731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhi tang updated IMPALA-11731:
------------------------------
Description:
In the minimal mode of the catalog server, only partition related information
needs to be protected by table locks when processing table updates. So I'm
thinking whether we can avoid the blocking of the table lock on the
catalog-update-gathering-thread in this way:
1. When an update arrives, we first determine which partitions will be affected
by the update after acquiring the table lock;
2. Then the catalog-update-gathering-thread collects the change information of
the partition. Because we already know the partitions that will be changed, we
do not need to hold table locks;
We do not expect the catalog-update-gathering-thread to be blocked because it
will block other unrelated updates, such as the problem described in
IMPALA-6671. It is necessary to deal with this problem in a more effective way.
was:
In the minimal mode of the catalog server, only partition related information
needs to be protected by table locks when processing table updates. So I'm
thinking whether we can avoid the blocking of the table lock on the collection
thread in this way:
1. When an update arrives, we first determine which partitions will be affected
by the update after obtaining the table lock;
2. Then the collection thread collects the change information of the partition.
Because we already know that those partitions will change, we do not need to
hold table locks;
We do not expect the collection thread to be blocked because it will block
other irrelevant updates, such as the problem described in IMPALA-6671. It is
necessary to deal with this problem in a more effective way.
> Consider avoiding table locks from blocking the
> catalog-update-gathering-thread when the catalog server's catalog_topic_mode
> is minimal.
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-11731
> URL: https://issues.apache.org/jira/browse/IMPALA-11731
> Project: IMPALA
> Issue Type: Improvement
> Reporter: zhi tang
> Priority: Major
>
> In the minimal mode of the catalog server, only partition related
> information needs to be protected by table locks when processing table
> updates. So I'm thinking whether we can avoid the blocking of the table lock
> on the catalog-update-gathering-thread in this way:
> 1. When an update arrives, we first determine which partitions will be
> affected by the update after acquiring the table lock;
> 2. Then the catalog-update-gathering-thread collects the change information
> of the partition. Because we already know the partitions that will be
> changed, we do not need to hold table locks;
> We do not expect the catalog-update-gathering-thread to be blocked because it
> will block other unrelated updates, such as the problem described in
> IMPALA-6671. It is necessary to deal with this problem in a more effective
> way.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
