Sergey Soldatov created HDDS-17873:
--------------------------------------

             Summary: StreamBlockInputStream should cancel, not half-close, a 
failed streaming read
                 Key: HDDS-17873
                 URL: https://issues.apache.org/jira/browse/HDDS-17873
             Project: Apache Ozone
          Issue Type: Bug
          Components: Ozone Datanode
    Affects Versions: 2.3.0
            Reporter: Sergey Soldatov
            Assignee: Sergey Soldatov


 When a streaming read fails, the client fails over to another replica. A 
common case is a stalled datanode that runs past 
ozone.client.stream.read.timeout. The client then closes the old stream the 
same way it closes a successful one: with a gRPC half-close (onCompleted()). A 
half-close does not end the call. It only tells the datanode that no more 
requests are coming.
  
  As a result:
  - The datanode handles one call's requests in order, so it sees the 
half-close only after the stuck request finishes. It then still sends the rest 
of that request's data to a client that has already moved on. With the default 
pre-read size, that can be about 32 MB.
  - On the client, the abandoned call and its reader stay alive until then, and 
the late data is queued in the abandoned reader. The streaming call has no gRPC 
deadline, so if the stuck request never finishes, nothing ends the call.
  - Each failover logs "Failed to setCompleted: already completed 
exceptionally" twice, with stack traces.

Positioned reads (pread) close a failed attempt the same way.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to