yanbin.zhang created HDFS-16186: ----------------------------------- Summary: Datanode kicks out hard disk logic optimization Key: HDFS-16186 URL: https://issues.apache.org/jira/browse/HDFS-16186 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 3.1.2 Environment: In the hadoop cluster, a certain hard disk in a certain Datanode has a problem, but the datanode of hdfs did not kick out the hard disk in time, causing the datanode to become a slow node Reporter: yanbin.zhang
2021-08-24 08:56:10,456 WARN datanode.DataNode (BlockSender.java:readChecksum(681)) - Could not read or failed to verify checksum for data at offset 113115136 for block BP-1801371083-x.x.x.x-1603704063698:blk_5635828768_4563943709 java.io.IOException: Input/output error at java.io.FileInputStream.readBytes(Native Method) at java.io.FileInputStream.read(FileInputStream.java:255) at org.apache.hadoop.hdfs.server.datanode.FileIoProvider$WrappedFileInputStream.read(FileIoProvider.java:876) at java.io.FilterInputStream.read(FilterInputStream.java:133) at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) at java.io.BufferedInputStream.read(BufferedInputStream.java:345) at java.io.DataInputStream.read(DataInputStream.java:149) at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:210) at org.apache.hadoop.hdfs.server.datanode.fsdataset.ReplicaInputStreams.readChecksumFully(ReplicaInputStreams.java:90) at org.apache.hadoop.hdfs.server.datanode.BlockSender.readChecksum(BlockSender.java:679) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:588) at org.apache.hadoop.hdfs.server.datanode.BlockSender.doSendBlock(BlockSender.java:803) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:750) at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.scanBlock(VolumeScanner.java:448) at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:558) at org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:633) 2021-08-24 08:56:11,121 WARN datanode.VolumeScanner (VolumeScanner.java:handle(292)) - Reporting bad BP-1801371083-x.x.x.x-1603704063698:blk_5635828768_4563943709 on /data11/hdfs/data -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org