[
https://issues.apache.org/jira/browse/HDFS-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13555399#comment-13555399
]
Alejandro Abdelnur commented on HDFS-4412:
------------------------------------------
Is the objective to be able to enforce IO throttling for the cluster or for
certain applications to be 'nice'? If the former this must be enforced on the
cluster side, not on the client side. If the later apps wanting to be 'nice'
could wrap the IO streams with throttling aware ones.
> Support HDFS IO throttling
> --------------------------
>
> Key: HDFS-4412
> URL: https://issues.apache.org/jira/browse/HDFS-4412
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Zhenxiao Luo
>
> When an applications upload/download files from/to HDFS clusters, it would be
> nice if the IO could be throttled so that they won't go beyond the specified
> maximum bandwidth.
> Two options to implement this IO throttling:
> #1. IO Throttling happens at the FSDataInputStream and FSDataOutputStream
> level.
> Add an IO Throttler to FSDataInputStream/FSDataOutputStram, and whenever an
> read/write happens, throttle it first(if throttler is set), then do the
> actual read/write.
> We may need to add new FileSystem apis to take an IO throttler as input
> parameter.
> #2. IO Throttling happens at the application level.
> Instead of changing the FSDataInputStream/FSDataOutputStream, all IO
> throttling is done at the application level.
> In this approach, FileSystem api remains unchanged.
> Either case, an IO throttler interface is needed, which has a:
> public void throttle(long numOfBytes);
> The current DataTransferThrottler could be an implementation of this IO
> throttler interface.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
