[
https://issues.apache.org/jira/browse/HDFS-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113329#comment-14113329
]
Arpit Agarwal edited comment on HDFS-6931 at 8/28/14 4:38 AM:
--------------------------------------------------------------
On restart each volume will be scanned and replicas under {{lazyPersist/}} will
be moved to their corresponding locations under {{finalized/}}.
We may end up with two replicas of the same block on different volumes, so we
use the following scheme to decide which replica to keep. See
{{BlockPoolSlice.resolveDuplicateReplicas}}.
# Prefer the replica with the higher generation stamp.
# If generation stamps are equal, prefer the replica with the larger on-disk
length.
# If on-disk length is the same, prefer the replica on persistent storage
volume.
# All other factors being equal, keep replica1.
The other replica is removed from the volumeMap and is deleted from its storage
volume.
Also added test cases.
was (Author: arpitagarwal):
On restart each volume will be scanned and replicas under {{lazyPersist/}} will
be moved to their corresponding locations under {{finalized/}}.
We may end up with two replicas of the same block on different volumes, so we
use the following scheme to decide which replica to keep. See
{{BlockPoolSlice.resolveDuplicateReplicas}}.
{code}
* Given two replicas, decide which one to keep. The preference is as
* follows:
* 1. Prefer the replica with the higher generation stamp.
* 2. If generation stamps are equal, prefer the replica with the
* larger on-disk length.
* 3. If on-disk length is the same, prefer the replica on persistent
* storage volume.
* 4. All other factors being equal, keep replica1.
*
* The other replica is removed from the volumeMap and is deleted from
* its storage volume.
{code}
Also added test cases.
> Move lazily persisted replicas to finalized directory on DN startup
> -------------------------------------------------------------------
>
> Key: HDFS-6931
> URL: https://issues.apache.org/jira/browse/HDFS-6931
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Fix For: HDFS-6581
>
> Attachments: HDFS-6931.01.patch
>
>
> On restart the DN should move replicas from the {{lazyPersist}} directory to
> {{finalized}}, and evict/unlink replicas of the same block from RAM disk.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
