[
https://issues.apache.org/jira/browse/SOLR-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18137273#comment-18137273
]
ASF subversion and git services commented on SOLR-18548:
--------------------------------------------------------
Commit a2626d839af4d63f78b067063c409fcef7b0a7fe in solr's branch
refs/heads/branch_10x from Boyi Dai
[ https://gitbox.apache.org/repos/asf?p=solr.git;h=a2626d839af ]
SOLR-18548: Fix overseer collection lock leak when marking an async task as
running fails (#5167)
(cherry picked from commit f489a30a31b1b97c7c8aa7a2c05c15d80114dc70)
> Overseer collection lock leaks when marking an async task as running fails
> --------------------------------------------------------------------------
>
> Key: SOLR-18548
> URL: https://issues.apache.org/jira/browse/SOLR-18548
> Project: Solr
> Issue Type: Bug
> Components: SolrCloud
> Affects Versions: 10.0, 9.11
> Reporter: Boyi Dai
> Assignee: Pierre Salagnac
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> OverseerTaskProcessor takes the lock for a task (collection, shard or
> replica) before storing the task’s async ID in the running map in ZooKeeper.
> If that {{runningMap.put}} fails, only {{NodeExistsException}} and
> {{InterruptedException}} release the lock. Any other {{KeeperException}} (for
> example {{{}OperationTimeoutException{}}}, or a {{ConnectionLossException}}
> after retries) or a {{RuntimeException}} goes to the outer loop’s handler
> with the lock still held.
> No Runner has been created at that point, so nothing ever releases the lock.
> Every later task that needs the same lock is blocked until the Overseer
> restarts, so a single transient ZooKeeper error during that write can stall
> all collection API operations on that collection.
> This has the same symptom as SOLR-17754 (a lock left held so tasks never
> run), but a different cause: SOLR-17754 fixed a race with the runner thread
> pool, while this happens when the ZooKeeper write fails before a Runner
> exists.
> Expected: the lock is released whenever the task is not handed to a Runner,
> so the task can be retried and other tasks for the same collection can
> proceed. A fix with a test is coming in a PR.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
