[
https://issues.apache.org/jira/browse/HDFS-11726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16027821#comment-16027821
]
Surendra Singh Lilhore commented on HDFS-11726:
-----------------------------------------------
I got the root cause for this.
SPS first give the preference to the local DN. If expected storage type is not
exist in source dn then it will select remote DN. We missed one check in
StoragePolicySatisfier#findSourceAndTargetToMove(). If replica with expected
storage type already exist in source DN then we should skip that DN.
For example:
Expected storage is : ARCHIVE
Source DN with storage : DN1(ARCHIVE), DN2(DISK).
We should not select DN1 for block movement because here already replica is
exist in ARCHIVE.
> [SPS] : StoragePolicySatisfier should not select same storage type as source
> and destination in same datanode.
> --------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-11726
> URL: https://issues.apache.org/jira/browse/HDFS-11726
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: HDFS-10285
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
>
> {code}
> 2017-04-30 16:12:28,569 [BlockMoverTask-0] INFO
> datanode.StoragePolicySatisfyWorker (Worker.java:moveBlock(248)) - Start
> moving block:blk_1073741826_1002 from src:127.0.0.1:41699 to
> destin:127.0.0.1:41699 to satisfy storageType, sourceStoragetype:ARCHIVE and
> destinStoragetype:ARCHIVE
> {code}
> {code}
> 2017-04-30 16:12:28,571 [DataXceiver for client /127.0.0.1:36428 [Replacing
> block BP-1409501412-127.0.1.1-1493548923222:blk_1073741826_1002 from
> 6c7aa66e-a778-43d5-89f6-053d5f6b35bc]] INFO datanode.DataNode
> (DataXceiver.java:replaceBlock(1202)) - opReplaceBlock
> BP-1409501412-127.0.1.1-1493548923222:blk_1073741826_1002 received exception
> org.apache.hadoop.hdfs.server.datanode.ReplicaAlreadyExistsException: Replica
> FinalizedReplica, blk_1073741826_1002, FINALIZED
> getNumBytes() = 1024
> getBytesOnDisk() = 1024
> getVisibleLength()= 1024
> getVolume() =
> /home/sachin/software/hadoop/HDFS-10285/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data7
> getBlockURI() =
> file:/home/sachin/software/hadoop/HDFS-10285/hadoop/hadoop-hdfs-project/hadoop-hdfs/target/test/data/dfs/data/data7/current/BP-1409501412-127.0.1.1-1493548923222/current/finalized/subdir0/subdir0/blk_1073741826
> already exists on storage ARCHIVE
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
