[
https://issues.apache.org/jira/browse/IMPALA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17908079#comment-17908079
]
Quanlong Huang commented on IMPALA-13631:
-----------------------------------------
CatalogServiceCatalog.renameTable() actually holds the catalogVersion writeLock
in the method:
{code:java}
public Pair<Table, Table> renameTable(
TTableName oldTableName, TTableName newTableName) {
// Remove the old table name from the cache and add the new table.
Db db = getDb(oldTableName.getDb_name());
if (db == null) return null;
versionLock_.writeLock().lock();
try {
Table oldTable =
removeTable(oldTableName.getDb_name(), oldTableName.getTable_name());
if (oldTable == null) return Pair.create(null, null);
return Pair.create(oldTable,
addIncompleteTable(newTableName.getDb_name(),
newTableName.getTable_name(),
oldTable.getTableType(), oldTable.getTableComment(),
oldTable.getCreateEventId()));
} finally {
versionLock_.writeLock().unlock();
}
}{code}
https://github.com/apache/impala/blob/0bbd2b684ddc7dcf8b6c16f1f7c6fab15291f782/fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java#L2744
We should revisit whether we need holding the lock in
CatalogOpExecutor.alterTableOrViewRename().
> alterTableOrViewRename shouldn't hold catalogVersionLock during external RPCs
> -----------------------------------------------------------------------------
>
> Key: IMPALA-13631
> URL: https://issues.apache.org/jira/browse/IMPALA-13631
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Major
>
> CatalogOpExecutor.alterTableOrViewRename() requires holding the
> catalogVersion writeLock, as the comment mentioned:
> {code:java}
> // RENAME is implemented as an ADD + DROP, so we need to execute it
> as we hold
> // the catalog lock.
> try {
> alterTableOrViewRename(tbl,
>
> TableName.fromThrift(params.getRename_params().getNew_table_name()),
> modification, wantMinimalResult, response, catalogTimeline);
> modification.validateInProgressModificationComplete();
> return;
> } finally {
> // release the version taken in the tryLock call above
> catalog_.getLock().writeLock().unlock();
> } {code}
> https://github.com/apache/impala/blob/0bbd2b684ddc7dcf8b6c16f1f7c6fab15291f782/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1221-L1232
> However, alterTableOrViewRename() triggers external RPCs, e.g. HMS
> alter_table RPC, which could hang due to external issues. Holding the
> catalogVersion writeLock blocks all other catalog operations, including all
> the read requests like getPartialCatalogObject or collecting catalog updates.
> This will impact the whole Impala cluster. Lots of queries will be blocked in
> the CREATED state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
