[
https://issues.apache.org/jira/browse/HDDS-17871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17871:
----------------------------------
Labels: pull-request-available (was: )
> Datanode gRPC connection flips writability on every positioned-read reply
> -------------------------------------------------------------------------
>
> Key: HDDS-17871
> URL: https://issues.apache.org/jira/browse/HDDS-17871
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Datanode
> Affects Versions: 2.3.0
> Reporter: Sergey Soldatov
> Assignee: Sergey Soldatov
> Priority: Minor
> Labels: pull-request-available
>
> Netty's default write buffer watermarks for a connection are 32 KiB (low) and
> 64 KiB (high). One positioned-read reply is bigger than the high watermark: a
> 64 KiB range plus the response's framing.
> Netty's HTTP/2 flow controller hands DATA to the connection only while the
> connection is writable
> ({{DefaultHttp2RemoteFlowController.writePendingBytes}}). So when the
> datanode flushes a reply:
> # the flow controller writes about 64 KiB, the connection turns unwritable,
> and the flow controller stops;
> # the connection writes those bytes to the socket; the buffer drops below 32
> KiB and the connection turns writable again (the event fires synchronously,
> inside the socket write);
> # {{Http2ConnectionHandler.channelWritabilityChanged}} sees pending data and
> flushes again, and the rest of the reply goes out in a separate socket write.
> Every positioned-read reply therefore costs two writability events through
> the pipeline, a second flow-controller pass, and an extra socket write. With
> N concurrent replies in one flush, the write is split into about N+1 socket
> writes and 2N writability events, instead of one write.
> gRPC's server creates its flow controller without a listener
> ({{NettyServerHandler}}), so a writability change does not walk the
> connection's streams; the cost is the split write above.
> h3. Measured
> Netty's HTTP/2 handler set up like gRPC's server
> ({{UniformStreamByteDistributor}}, 1 MiB window), with an {{EmbeddedChannel}}
> standing in for a socket that accepts all bytes.
> ||Watermark||Replies||Events while sending||
> |32K / 64K (default)|1|unwritable, write 65381 B, writable, write 1233 B|
> |32K / 64K (default)|4|4 × (unwritable, write ~64 KiB, writable), then write
> 4932 B|
> |512K / 1M|1|one write of 66614 B|
> |512K / 1M|4|one write of 266456 B|
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]