boglesby commented on PR #7610:
URL: https://github.com/apache/geode/pull/7610#issuecomment-1113500376
Its definitely a corner case.
Here is the sequence:
1. The lock requesting server requests a lock
2. Before the grantor server processes that request, the lock requesting
server leaves the distributed system
3. The DistributionManager in the grantor server removes the member and logs
the `Member gracefully left` message
4. The DLockGrantor in the grantor server begins to process the lock request
5. The DLockGrantor's membersDepartedTime map is updated with an entry for
that member
6. The grantAndRespondToRequest method calls isCurrentMember which returns
true because the member is still in the view
7. The DLockGrantor grants the lock
8. The view is updated to remove the departed member
At the end of this sequence, the lock is held by a member who has departed.
Here is some logging that shows the sequence from the grantor server's point
of view.
Original view:
```
[vm1] [warn 2022/04/28 12:57:27.594 PDT server-1 <unicast
receiver,host-47928> tid=0x27] XXX MembershipView.<init>
view=View[host(locator-0:46041:locator)<ec><v0>:41001|2] members:
[host(locator-0:46041:locator)<ec><v0>:41001,
host(server-1:46042)<v1>:41002{lead}, host(server-2:46043)<v2>:41003]
```
Lock requesting server departs:
```
[vm1] [debug 2022/04/28 12:57:29.201 PDT server-1 <Pooled High Priority
Message Processor 3> tid=0x4f] DistributionManager: removing member
<host(server-2:46043)<v2>:41003>; crashed false; reason = shutdown message
received
[vm1] [info 2022/04/28 12:57:29.201 PDT server-1 <Pooled High Priority
Message Processor 3> tid=0x4f] Member at host(server-2:46043)<v2>:41003
gracefully left the distributed cache: shutdown message received
```
The membersDepartedTime map is updated:
```
[vm1] [info 2022/04/28 12:57:29.221 PDT server-1 <Pooled Waiting Message
Processor 1> tid=0x2f] XXX DLockGrantor.recordMemberDepartedTime recorded
membersDepartedTime owner=host(server-2:46043)<v2>:41003;
currentTime=1651175849221; membersDepartedTimeSize=1;
membersDepartedTime={host(server-2:46043)<v2>:41003=1651175849221}
```
The grantAndRespondToRequest method checks is current member which returns
true:
```
[vm1] [warn 2022/04/28 12:57:29.259 PDT server-1 <Pooled Waiting Message
Processor 1> tid=0x2f] XXX DLockGrantToken.grantAndRespondToRequest
currentMember=host(server-2:46043)<v2>:41003; isCurrentMember=true
```
The DLockGrantor grants the lock:
```
[vm1] [debug 2022/04/28 12:57:29.260 PDT server-1 <Pooled Waiting Message
Processor 2> tid=0x4f] Sending (DLockRequestProcessor.DLockResponseMessage
responding GRANT; serviceName=__PRLS(version 3);
objectName=testRequestDistributedLockWhileClosingCache_lockName;
responseCode=0; keyIfFailed=null; leaseExpireTime=9223372036854775807;
processorId=21; lockId=21) to 1 peers ([host(server-2:46043)<v2>:41003]) via
tcp/ip
```
The view is updated:
```
[vm1] [warn 2022/04/28 12:57:29.532 PDT server-1 <unicast
receiver,host-47928> tid=0x27] XXX MembershipView.<init>
view=View[host(locator-0:46041:locator)<ec><v0>:41001|3] members:
[host(locator-0:46041:locator)<ec><v0>:41001,
host(server-1:46042)<v1>:41002{lead}] shutdown:
[host(server-2:46043)<v2>:41003]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
