[
https://issues.apache.org/jira/browse/HBASE-30393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huginn updated HBASE-30393:
---------------------------
Description:
SimpleRpcServer can retain a stale response timestamp on a connection after a
queued RPC response has been fully written. If a later response on the same
connection cannot make progress, the stale timestamp can make the connection
appear to have been blocked longer than the current response.
When SimpleRpcServerResponder.processResponse writes a response in multiple
attempts, it records lastSentTime while the response is incomplete. The
timestamp is not cleared when the response completes, and a non-blocking write
that makes no progress can refresh the timestamp. The responder purge path can
then close the connection based on stale response state, causing RPC failures
and retries.
was:
SimpleRpcServer can retain a stale response timestamp on a connection after a
queued RPC response has been fully written. If a later response on the same
connection cannot make progress, the stale timestamp can make the connection
appear to have been blocked longer than the current response.
When SimpleRpcServerResponder.processResponse writes a response in multiple
attempts, it records lastSentTime while the response is incomplete. The
timestamp is not cleared when the response completes, and a non-blocking write
that makes no progress can refresh the timestamp. The responder purge path can
then close the connection based on stale response state, causing RPC failures
and retries.
The fix is to clear lastSentTime when the current response is fully written,
preserve or update it only while an incomplete response remains, skip purge for
connections with an empty response queue, and make the timestamp safely visible
between the responder and purge paths.
Reproduction and validation:
* Queue a response that is partially written, complete it, then queue a
response whose non-blocking write makes no progress on the same connection.
* Before the fix, TestSimpleRpcServerResponder reproduces stale timestamp
retention, timestamp refresh on a stalled write, and purge of a connection with
an empty response queue.
* After the fix, all three regression tests pass; the related IPC regression
tests and the hbase-server package build also pass.
> Clear stale RPC response timestamps after responses complete
> ------------------------------------------------------------
>
> Key: HBASE-30393
> URL: https://issues.apache.org/jira/browse/HBASE-30393
> Project: HBase
> Issue Type: Bug
> Components: IPC/RPC
> Affects Versions: 2.4.11
> Reporter: huginn
> Assignee: huginn
> Priority: Major
> Labels: pull-request-available
>
> SimpleRpcServer can retain a stale response timestamp on a connection after a
> queued RPC response has been fully written. If a later response on the same
> connection cannot make progress, the stale timestamp can make the connection
> appear to have been blocked longer than the current response.
> When SimpleRpcServerResponder.processResponse writes a response in multiple
> attempts, it records lastSentTime while the response is incomplete. The
> timestamp is not cleared when the response completes, and a non-blocking
> write that makes no progress can refresh the timestamp. The responder purge
> path can then close the connection based on stale response state, causing RPC
> failures and retries.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
