[
https://issues.apache.org/jira/browse/HDFS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403894#comment-13403894
]
Harsh J commented on HDFS-3475:
-------------------------------
The value for the multiplier we used was 100. This gave us a good boost, but
was tried on a cluster that had a very good network gear.
Yes I do feel the default of 2 is too low today. However, it is low to prevent
unnecessary replication in face of long-taking DN restarts and for that reason
it does well not to consume too much network. Hence I had left in the defaults.
It has to be raised only for fast networks, otherwise the reads/writes may get
choked. So I do not think we need to raise it, given the variance of users who
use Hadoop.
Let me know if you feel otherwise Eli.
P.s. The commit HDFS build failed cause of HDFS-RAID project issues, not HDFS.
No alarm, hence.
> Make the replication monitor multipliers configurable
> -----------------------------------------------------
>
> Key: HDFS-3475
> URL: https://issues.apache.org/jira/browse/HDFS-3475
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.0.0-alpha
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Trivial
> Fix For: 2.0.1-alpha
>
> Attachments: HDFS-3475.patch, HDFS-3475.patch, HDFS-3475.patch
>
>
> BlockManager currently hardcodes the following two constants:
> {code}
> private static final int INVALIDATE_WORK_PCT_PER_ITERATION = 32;
> private static final int REPLICATION_WORK_MULTIPLIER_PER_ITERATION = 2;
> {code}
> These are used to throttle/limit the amount of deletion and
> replication-to-other-DN work done per heartbeat interval of a live DN.
> Not many have had reasons to want these changed so far but there have been a
> few requests I've faced over the past year from a variety of clusters I've
> helped maintain. I think with the improvements in disks and network thats
> already started to be rolled out in production environments out there,
> changing these may start making sense to some.
> Lets at least make it advanced-configurable with proper docs that warn
> adequately, with the defaults being what they are today. With hardcodes, it
> comes down to a recompile for admins, which is not something they may like.
> Please let me know your thoughts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
