[
https://issues.apache.org/jira/browse/HDFS-7270?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14261358#comment-14261358
]
Arpit Agarwal commented on HDFS-7270:
-------------------------------------
Can we rename the Jira to something like _"Add congestion signaling capability
to DataNode write protocol"_ since this change does not implement congestion
control.
The approach looks fine. Minor comments:
# StatusFormat values can take StatusFormat.BITS in constructor instead of
StatusFormat. Then you don't need the null check. See
{{INodeFile.HeaderFormat}}.
# Why do we need the SUPPORTED2 value?
# Add a comment to enum {{Status}} in DataTransfer.proto that it is a 4-bit
value? It is close to full so if someone adds a few more status codes it can
overflow.
# Would you add a short Javadoc for DataNode.getECN since it will need to be
extended in the next step?
Can we add test cases to ensure ECN.SUPPORTED and ECN.CONGESTED are propagated?
> Implementing congestion control in writing pipeline
> ---------------------------------------------------
>
> Key: HDFS-7270
> URL: https://issues.apache.org/jira/browse/HDFS-7270
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Haohui Mai
> Assignee: Haohui Mai
> Attachments: HDFS-7270.000.patch, HDFS-7270.001.patch
>
>
> When a client writes to HDFS faster than the disk bandwidth of the DNs, it
> saturates the disk bandwidth and put the DNs unresponsive. The client only
> backs off by aborting / recovering the pipeline, which leads to failed writes
> and unnecessary pipeline recovery.
> This jira proposes to add explicit congestion control mechanisms in the
> writing pipeline.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
