[
https://issues.apache.org/jira/browse/HDFS-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12738652#action_12738652
]
Konstantin Shvachko commented on HDFS-517:
------------------------------------------
- {{BlockInfoUnderConstruction}} should be a subclass of {{BlockInfo}}. It
should contain an array of {{locations}} reflecting the write pipeline order,
and replica states.
- When a new block is allocated it is created as a
{{BlockInfoUnderConstruction}}. When the block changes its state to
{{COMPLETE}} the {{BlockInfoUnderConstruction}} is replaced by regular
{{BlockInfo}} in the blocksMap, the file inode, and the data-node block list.
- Currently we do not maintain replication of blocks under construction so
other maps like {{UnderReplicatedBlocks}} or {{excessReplicateMap}} would not
contain these blocks and therefore do need to be updated.
- {{BlockInfoUnderConstruction.locations}} should be used to maintain the
pipeline and for block recovery, but not for reading. So locations remain
invisible for readers until the replicas are reported by data-nodes. This is
consistent with current behavior, and should change later when the new read is
implemented.
- When a replica is reported by the data-node the block is included into the
list of blocks belonging to the corresponding {{DatanodeDescriptor}} and thus
becomes readable.
- The {{locations}} and {{replicaStates}} are not persistent fields (consistent
with current implementation). When name-node restarts it sets the block state
to {{UNDER_RECOVERY}} and waits for block recovery according to the append
design.
- {{INodeFileUnderConstruction.targets}} field becomes redundant.
> Introduce BlockInfoUnderConstruction to reflect block replica states while
> writing.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-517
> URL: https://issues.apache.org/jira/browse/HDFS-517
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: name-node
> Affects Versions: 0.21.0
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Fix For: Append Branch
>
>
> Currently when a block is created its locations are stored in
> {{INodeFileUnderConstruction.targets}}, which correspond to the last
> allocated block. With the new append design we will need to keep track of
> block replicas for several blocks rather than just the last one.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
