[ https://issues.apache.org/jira/browse/HDFS-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388983#comment-15388983 ]
Yongjun Zhang commented on HDFS-10652: -------------------------------------- Hi [~vinayrpet], Thanks for the new rev. I have two comments: 1. Improved msg printing in Datanode.java and FsDatasetImpl.java, where I added logging of current datanode. 2. I change the condition {{replicaInfo.getBytesAcked() < replicaInfo.getBytesOnDisk()}} to {code} if ((replicaInfo.getBytesOnDisk() / chunkSize) - (replicaInfo.getBytesAcked() / chunkSize) >= 1) { {code} to make sure that not only the bytesAcked is smaller than the butesOnDisk, but the latter has to be equal to or bigger than the chunk end. Uploaded rev 005 to reflect these comments. BTW, I still saw a little flakiness, probably one out of 10 times, I would see that the test failed, reason is: {code} unning org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 79.142 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery testPipelineRecoveryWithTransferBlock(org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery) Time elapsed: 21.456 sec <<< ERROR! org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-537180841-127.0.0.1-1469127392654:blk_1073741825_1002 file=/f at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1039) at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:654) at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:946) at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1001) at java.io.DataInputStream.read(DataInputStream.java:100) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:88) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62) at org.apache.hadoop.hdfs.DFSTestUtil.readFileBuffer(DFSTestUtil.java:354) at org.apache.hadoop.hdfs.DFSTestUtil.readFile(DFSTestUtil.java:346) at org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery.testPipelineRecoveryWithTransferBlock(TestClientProtocolForPipelineRecovery.java:588) {code} Thanks. > Add a unit test for HDFS-4660 > ----------------------------- > > Key: HDFS-10652 > URL: https://issues.apache.org/jira/browse/HDFS-10652 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs > Reporter: Yongjun Zhang > Assignee: Vinayakumar B > Attachments: HDFS-10652-002.patch, HDFS-10652.001.patch, > HDFS-10652.003.patch, HDFS-10652.004.patch, HDFS-10652.005.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org