[
https://issues.apache.org/jira/browse/ACCUMULO-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916304#comment-13916304
]
ASF subversion and git services commented on ACCUMULO-2422:
-----------------------------------------------------------
Commit 7eeff02c7cf883765a33575a19d208be30e1e17c in accumulo's branch
refs/heads/1.6.0-SNAPSHOT from [~bhavanki]
[ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7eeff02 ]
ACCUMULO-2422 Refine renewal of master lock watcher
The first commit for ACCUMULO-2422 succeeds in renewing the watch on another
master's lock
node when needed. This commit refines the solution:
- The renewal was happening even after the master is able to acquire the lock.
This led to a
spurious log error message. This commit skips renewing the watch in that case.
- If the renewal returns a null status, meaning the other master's lock node
disappeared, the
master now immediately tries again to acquire the lock. This matches watch
establishment in
other areas.
A lot of logging at the trace level was added to ZooLock to assist future
troubleshooting.
> Backup master can miss acquiring lock when primary exits
> --------------------------------------------------------
>
> Key: ACCUMULO-2422
> URL: https://issues.apache.org/jira/browse/ACCUMULO-2422
> Project: Accumulo
> Issue Type: Bug
> Components: fate, master
> Affects Versions: 1.5.1
> Reporter: Bill Havanki
> Assignee: Bill Havanki
> Priority: Critical
> Labels: failover, locking
> Fix For: 1.6.0, 1.5.2
>
>
> While running randomwalk tests with agitation for the 1.5.1 release, I've
> seen situations where a backup master that is eligible to grab the master
> lock continues to wait. When this condition arises and the other master
> restarts, both wait for the lock without success.
> I cannot reproduce the problem reliably, and I think more investigation is
> needed to see what circumstances could be causing the problem.
> h3. Diagnosis and Work Around
> This failure condition can occur on start up and on backup/active failover of
> the Master role. If the follow log entry is the final entry on all Master
> logs you should restart all Master roles, staggering by a few seconds.
> {noformat}
> [master.Master] INFO : trying to get master lock
> {noformat}
> If starting a cluster with multiple Master roles, you should stagger Master
> role starts by a few seconds.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
