[
https://issues.apache.org/jira/browse/IMPALA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17946243#comment-17946243
]
ASF subversion and git services commented on IMPALA-13631:
----------------------------------------------------------
Commit ab92a300fc9363895418690b87b4a73df7a7202d in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ab92a300f ]
IMPALA-13631: alterTableOrViewRename shouldn't hold catalog versionLock during
external RPCs
Catalog versionLock is a lock used to synchronize reads/writes of
catalogVersion. It can be used to perform atomic bulk catalog operations
since catalogVersion cannot change externally while the lock is being
held. All other catalog operations will be blocked if the current thread
holds the lock. So it shouldn't be held for a long time, especially when
the current thread is invoking external RPCs for a table.
CatalogOpExecutor.alterTable() is one place that could hold the lock for
a long time. If the ALTER operation is a RENAME, it holds the lock until
alterTableOrViewRename() finishes. HMS RPCs are invoked in this method
to perform the operation, which might take an unpredictive time. The
motivation of holding this lock is that RENAME is implemented as an DROP
+ ADD in the catalog cache. So this operation can be atomic. However,
that doesn't mean we need the lock before operating the cache in
CatalogServiceCatalog.renameTable(). We actually acquires the lock again
in this method. So no need to keep holding the lock when invoking HMS
RPCs.
This patch removes holding the lock in alterTableOrViewRename().
Tests
- Added e2e test for concurrent rename operations.
- Also added some rename operations in test_concurrent_ddls.py
Change-Id: Ie5f443b1e167d96024b717ce70ca542d7930cb4b
Reviewed-on: http://gerrit.cloudera.org:8080/22789
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
> alterTableOrViewRename shouldn't hold catalogVersionLock during external RPCs
> -----------------------------------------------------------------------------
>
> Key: IMPALA-13631
> URL: https://issues.apache.org/jira/browse/IMPALA-13631
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
>
> CatalogOpExecutor.alterTableOrViewRename() requires holding the
> catalogVersion writeLock, as the comment mentioned:
> {code:java}
> // RENAME is implemented as an ADD + DROP, so we need to execute it
> as we hold
> // the catalog lock.
> try {
> alterTableOrViewRename(tbl,
>
> TableName.fromThrift(params.getRename_params().getNew_table_name()),
> modification, wantMinimalResult, response, catalogTimeline);
> modification.validateInProgressModificationComplete();
> return;
> } finally {
> // release the version taken in the tryLock call above
> catalog_.getLock().writeLock().unlock();
> } {code}
> https://github.com/apache/impala/blob/0bbd2b684ddc7dcf8b6c16f1f7c6fab15291f782/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1221-L1232
> However, alterTableOrViewRename() triggers external RPCs, e.g. HMS
> alter_table RPC, which could hang due to external issues. Holding the
> catalogVersion writeLock blocks all other catalog operations, including all
> the read requests like getPartialCatalogObject or collecting catalog updates.
> This will impact the whole Impala cluster. Lots of queries will be blocked in
> the CREATED state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
