[ https://issues.apache.org/jira/browse/HDFS-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113329#comment-14113329 ]
Arpit Agarwal edited comment on HDFS-6931 at 8/28/14 4:38 AM: -------------------------------------------------------------- On restart each volume will be scanned and replicas under {{lazyPersist/}} will be moved to their corresponding locations under {{finalized/}}. We may end up with two replicas of the same block on different volumes, so we use the following scheme to decide which replica to keep. See {{BlockPoolSlice.resolveDuplicateReplicas}}. # Prefer the replica with the higher generation stamp. # If generation stamps are equal, prefer the replica with the larger on-disk length. # If on-disk length is the same, prefer the replica on persistent storage volume. # All other factors being equal, keep replica1. The other replica is removed from the volumeMap and is deleted from its storage volume. Also added test cases. was (Author: arpitagarwal): On restart each volume will be scanned and replicas under {{lazyPersist/}} will be moved to their corresponding locations under {{finalized/}}. We may end up with two replicas of the same block on different volumes, so we use the following scheme to decide which replica to keep. See {{BlockPoolSlice.resolveDuplicateReplicas}}. {code} * Given two replicas, decide which one to keep. The preference is as * follows: * 1. Prefer the replica with the higher generation stamp. * 2. If generation stamps are equal, prefer the replica with the * larger on-disk length. * 3. If on-disk length is the same, prefer the replica on persistent * storage volume. * 4. All other factors being equal, keep replica1. * * The other replica is removed from the volumeMap and is deleted from * its storage volume. {code} Also added test cases. > Move lazily persisted replicas to finalized directory on DN startup > ------------------------------------------------------------------- > > Key: HDFS-6931 > URL: https://issues.apache.org/jira/browse/HDFS-6931 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: datanode > Reporter: Arpit Agarwal > Assignee: Arpit Agarwal > Fix For: HDFS-6581 > > Attachments: HDFS-6931.01.patch > > > On restart the DN should move replicas from the {{lazyPersist}} directory to > {{finalized}}, and evict/unlink replicas of the same block from RAM disk. -- This message was sent by Atlassian JIRA (v6.2#6252)