Lohit Vijayarenu created HDFS-5499:
--------------------------------------
Summary: Provide way to throttle per FileSystem read/write
bandwidth
Key: HDFS-5499
URL: https://issues.apache.org/jira/browse/HDFS-5499
Project: Hadoop HDFS
Issue Type: Improvement
Reporter: Lohit Vijayarenu
In some cases it might be worth to throttle read/writer bandwidth on per JVM
basis so that clients do not spawn too many thread and start data movement
causing other JVMs to starve. Ability to throttle read/write bandwidth on per
FileSystem would help avoid such issues.
Challenge seems to be how well this can be fit into FileSystem code. If one
enables throttling around FileSystem APIs, then any hidden data transfer within
cluster using them might also be affected. Eg. copying job jar during job
submission, localizing resources for distributed cache and such.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
