[ https://issues.apache.org/jira/browse/HDFS-7877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15864638#comment-15864638 ]
Manoj Govindassamy commented on HDFS-7877: ------------------------------------------ Thanks [~mingma]. Got it, when you club this with Upgrade Domain, the impact is not that severe. I will make the following change for the Maintenance Min Replication range validation check. {noformat} --- a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java @@ -484,12 +484,12 @@ public BlockManager(final Namesystem namesystem, boolean haEnabled, + DFSConfigKeys.DFS_NAMENODE_MAINTENANCE_REPLICATION_MIN_KEY + " = " + minMaintenanceR + " < 0"); } - if (minMaintenanceR > minR) { + if (minMaintenanceR > defaultReplication) { throw new IOException("Unexpected configuration parameters: " + DFSConfigKeys.DFS_NAMENODE_MAINTENANCE_REPLICATION_MIN_KEY + " = " + minMaintenanceR + " > " - + DFSConfigKeys.DFS_NAMENODE_REPLICATION_MIN_KEY - + " = " + minR); + + DFSConfigKeys.DFS_REPLICATION_DEFAULT + + " = " + defaultReplication); } {noformat} bq. the transition policy from ENTERING_MAINTENANCE to IN_MAINTENANCE will become the # of live replicas >= min(dfs.namenode.maintenance.replication.min, replication factor). But, the transition from ENTERIN_MM to IN_MM that is happening {{DecommissionManager#Monitor#check}} which in-turn calls {{DecommissionManager#isSufficient}} looks ok to me. Because, we allow files to be created with custom block replication count say 1, which can be lesser than the default dfs.replication=3. And, since we should not be counting in the Maintenance Replicas, the formula is, as it exists currently {noformat} expectedRedundancy = file_block_replication_count=1 or the default_replication_cont=3 Math.max( expectedRedundancy - numberReplicas.maintenanceReplicas(), getMinMaintenanceStorageNum(block)); {noformat} Let me know if I am missing something. Thanks. --- related code snippets ---- {noformat} /** * Checks whether a block is sufficiently replicated/stored for * decommissioning. For replicated blocks or striped blocks, full-strength * replication or storage is not always necessary, hence "sufficient". * @return true if sufficient, else false. */ private boolean isSufficient(BlockInfo block, BlockCollection bc, NumberReplicas numberReplicas, boolean isDecommission) { if (blockManager.hasEnoughEffectiveReplicas(block, numberReplicas, 0)) { // Block has enough replica, skip LOG.trace("Block {} does not need replication.", block); return true; } .. .. .. // Check if the number of live + pending replicas satisfies // the expected redundancy. boolean hasEnoughEffectiveReplicas(BlockInfo block, NumberReplicas numReplicas, int pendingReplicaNum) { int required = getExpectedLiveRedundancyNum(block, numReplicas); int numEffectiveReplicas = numReplicas.liveReplicas() + pendingReplicaNum; return (numEffectiveReplicas >= required) && (pendingReplicaNum > 0 || isPlacementPolicySatisfied(block)); } // Exclude maintenance, but make sure it has minimal live replicas // to satisfy the maintenance requirement. public short getExpectedLiveRedundancyNum(BlockInfo block, NumberReplicas numberReplicas) { final short expectedRedundancy = getExpectedRedundancyNum(block); return (short) Math.max(expectedRedundancy - numberReplicas.maintenanceReplicas(), getMinMaintenanceStorageNum(block)); } {noformat} > Support maintenance state for datanodes > --------------------------------------- > > Key: HDFS-7877 > URL: https://issues.apache.org/jira/browse/HDFS-7877 > Project: Hadoop HDFS > Issue Type: New Feature > Components: datanode, namenode > Reporter: Ming Ma > Assignee: Ming Ma > Attachments: HDFS-7877-2.patch, HDFS-7877.patch, > Supportmaintenancestatefordatanodes-2.pdf, > Supportmaintenancestatefordatanodes.pdf > > > This requirement came up during the design for HDFS-7541. Given this feature > is mostly independent of upgrade domain feature, it is better to track it > under a separate jira. The design and draft patch will be available soon. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org