[ https://issues.apache.org/jira/browse/HDFS-3475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426186#comment-13426186 ]
Adam Muise commented on HDFS-3475: ---------------------------------- Note on this value: while we tested at 100, this may be too high for a cluster under even moderate workload. The cluster in question had very powerful nodes and dual-bonded 10Gb interfaces. We also had to increase the DataNode memory in the range of 4-6Gb. You may choose to start the value at 10 and go up based on your available memory for the Datanode, I/O, and network capacity. > Make the replication monitor multipliers configurable > ----------------------------------------------------- > > Key: HDFS-3475 > URL: https://issues.apache.org/jira/browse/HDFS-3475 > Project: Hadoop HDFS > Issue Type: Improvement > Affects Versions: 2.0.0-alpha > Reporter: Harsh J > Assignee: Harsh J > Priority: Trivial > Fix For: 2.1.0-alpha > > Attachments: HDFS-3475.patch, HDFS-3475.patch, HDFS-3475.patch > > > BlockManager currently hardcodes the following two constants: > {code} > private static final int INVALIDATE_WORK_PCT_PER_ITERATION = 32; > private static final int REPLICATION_WORK_MULTIPLIER_PER_ITERATION = 2; > {code} > These are used to throttle/limit the amount of deletion and > replication-to-other-DN work done per heartbeat interval of a live DN. > Not many have had reasons to want these changed so far but there have been a > few requests I've faced over the past year from a variety of clusters I've > helped maintain. I think with the improvements in disks and network thats > already started to be rolled out in production environments out there, > changing these may start making sense to some. > Lets at least make it advanced-configurable with proper docs that warn > adequately, with the defaults being what they are today. With hardcodes, it > comes down to a recompile for admins, which is not something they may like. > Please let me know your thoughts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira