[
https://issues.apache.org/jira/browse/HDFS-15?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jitendra Nath Pandey updated HDFS-15:
-------------------------------------
Attachment: HDFS-15.patch
> There was a time when the Namenode was littered with plenty of adhoc data
> structures, each for its own purpose. There was an effort to consolidate the
> functionality of these data structures into a smaller set. I am not against
> this patch, but is it really difficult to integrate this new data structure
> into neededReplication as explained in your first proposal?
It is not difficult but it will make the code confusing. At many places it is
assumed that a block in neededReplications is strictly under-replicated and
that will make the logic complicated. Code will be cleaner if we use strict
definitions of neededReplications and neededReplicationsForRacks queues.
Also it will include significant changes to UnderReplicatedBlocks.java. If we
use seperate queue all the code changes are confined to BlockManager.java
The patch is attached.
> All replicas of a block end up on only 1 rack
> ---------------------------------------------
>
> Key: HDFS-15
> URL: https://issues.apache.org/jira/browse/HDFS-15
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: Hairong Kuang
> Assignee: Jitendra Nath Pandey
> Priority: Critical
> Attachments: HDFS-15.patch
>
>
> HDFS replicas placement strategy guarantees that the replicas of a block
> exist on at least two racks when its replication factor is greater than one.
> But fsck still reports that the replicas of some blocks end up on one rack.
> The cause of the problem is that decommission and corruption handling only
> check the block's replication factor but not the rack requirement. When an
> over-replicated block loses a replica due to decomission, corruption, or
> heartbeat lost, namenode does not take any action to guarantee that remaining
> replicas are on different racks.
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
