[
https://issues.apache.org/jira/browse/HDFS-15773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David resolved HDFS-15773.
--------------------------
Resolution: Information Provided
> NameNode allocate five DataNodes for a small EC file (606 bytes) in log file
> ----------------------------------------------------------------------------
>
> Key: HDFS-15773
> URL: https://issues.apache.org/jira/browse/HDFS-15773
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: David
> Priority: Major
>
> For testing Erasure Coding, I put a small file "hosts4" (606 bytes) to a EC
> enabled HDFS directory.
> . In the output of the fsck on this file, there are three replicas which is
> not surprise cause it's small than 1M.
>
> _/ec1/hosts4 606 bytes, erasure-coded: policy=RS-3-2-1024k, 1 block(s): OK_
> _0. BP-1329090504-172.25.37.132-1610344959454:blk_-9223372036854775184_1133
> len=606 Live_repl=3
> [blk_-9223372036854775184:DatanodeInfoWithStorage[172.25.34.135:9866,DS-b82b3dd4-cf57-4be6-9a0e-21089cf01d17,DISK],
>
> blk_-9223372036854775181:DatanodeInfoWithStorage[172.25.35.145:9866,DS-ef58c1c4-b54b-4321-96e5-eb39dd8e21cd,DISK],
>
> blk_-9223372036854775180:DatanodeInfoWithStorage[172.25.40.139:9866,DS-a160a4ae-ac88-4103-b19c-dc0f3fc0f861,DISK]]_
> . But in the NameNode log file, there were five DataNodes have been selected.
> However, only three DNs reported FINALIZED status, and there is a gap on the
> block IDs.
>
> _2021-01-12 11:12:54,204 DEBUG org.apache.hadoop.hdfs.StateChange: DIR*
> FSDirectory.addBlock: /ec1/hosts4._COPYING_ with
> blk_-9223372036854775184_1133 block is added to the in-memory file system_
> _2021-01-12 11:12:54,204 INFO org.apache.hadoop.hdfs.StateChange: BLOCK*
> allocate blk_-9223372036854775184_1133, replicas=172.25.34.135:9866,
> 172.25.38.155:9866, 172.25.37.147:9866, 172.25.35.145:9866,
> 172.25.40.139:9866 for /ec1/hosts4._COPYING__
>
> _2021-01-12 11:12:54,288 DEBUG
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Reported block
> blk_-9223372036854775184_1133 on 172.25.34.135:9866 size 606 replicaState =
> FINALIZED_
> _2021-01-12 11:12:54,291 DEBUG
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Reported block
> blk_-9223372036854775181_1133 on 172.25.35.145:9866 size 606 replicaState =
> FINALIZED_
> _2021-01-12 11:12:54,293 DEBUG
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Reported block
> blk_-9223372036854775180_1133 on 172.25.40.139:9866 size 606 replicaState =
> FINALIZED_
>
> This issue can be easily reproduced in HADOOP 3.0.x and 3.1.x.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
