[
https://issues.apache.org/jira/browse/IMPALA-13631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17947576#comment-17947576
]
ASF subversion and git services commented on IMPALA-13631:
----------------------------------------------------------
Commit 74bd0832ed20aa0c2d1ef35428b2337b973cbcf4 in impala's branch
refs/heads/master from Michael Smith
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=74bd0832e ]
IMPALA-13631: (Addendum) Test slow concurrent alters
Adds a test that multiple slow concurrent alters complete in parallel
rather than being executed sequentially.
The test creates tables, and ensures Impala's catalog is up-to-date for
their creation so that starting ALTER TABLE will be fast. Then starts
two ALTER TABLE RENAMES asynchronously - with debug_action ensuring each
takes at least 5 seconds - and waits for them to finish.
Verifies that concurrent alters are no longer blocked on "Got catalog
version read lock" and complete in less than 10 seconds.
Change-Id: I87d077aaa295943a16e6da60a2755dd289f3a132
Reviewed-on: http://gerrit.cloudera.org:8080/22804
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> alterTableOrViewRename shouldn't hold catalogVersionLock during external RPCs
> -----------------------------------------------------------------------------
>
> Key: IMPALA-13631
> URL: https://issues.apache.org/jira/browse/IMPALA-13631
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.5.0
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 5.0.0
>
>
> CatalogOpExecutor.alterTableOrViewRename() requires holding the
> catalogVersion writeLock, as the comment mentioned:
> {code:java}
> // RENAME is implemented as an ADD + DROP, so we need to execute it
> as we hold
> // the catalog lock.
> try {
> alterTableOrViewRename(tbl,
>
> TableName.fromThrift(params.getRename_params().getNew_table_name()),
> modification, wantMinimalResult, response, catalogTimeline);
> modification.validateInProgressModificationComplete();
> return;
> } finally {
> // release the version taken in the tryLock call above
> catalog_.getLock().writeLock().unlock();
> } {code}
> https://github.com/apache/impala/blob/0bbd2b684ddc7dcf8b6c16f1f7c6fab15291f782/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java#L1221-L1232
> However, alterTableOrViewRename() triggers external RPCs, e.g. HMS
> alter_table RPC, which could hang due to external issues. Holding the
> catalogVersion writeLock blocks all other catalog operations, including all
> the read requests like getPartialCatalogObject or collecting catalog updates.
> This will impact the whole Impala cluster. Lots of queries will be blocked in
> the CREATED state.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
