[
https://issues.apache.org/jira/browse/HDFS-10652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388983#comment-15388983
]
Yongjun Zhang commented on HDFS-10652:
--------------------------------------
Hi [~vinayrpet],
Thanks for the new rev. I have two comments:
1. Improved msg printing in Datanode.java and FsDatasetImpl.java, where I added
logging of current datanode.
2. I change the condition {{replicaInfo.getBytesAcked() <
replicaInfo.getBytesOnDisk()}} to
{code}
if ((replicaInfo.getBytesOnDisk() / chunkSize) -
(replicaInfo.getBytesAcked() / chunkSize) >= 1) {
{code}
to make sure that not only the bytesAcked is smaller than the butesOnDisk, but
the latter has to be equal to or bigger than the chunk end.
Uploaded rev 005 to reflect these comments.
BTW, I still saw a little flakiness, probably one out of 10 times, I would see
that the test failed, reason is:
{code}
unning org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 79.142 sec <<<
FAILURE! - in org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery
testPipelineRecoveryWithTransferBlock(org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery)
Time elapsed: 21.456 sec <<< ERROR!
org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block:
BP-537180841-127.0.0.1-1469127392654:blk_1073741825_1002 file=/f
at
org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1039)
at
org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:654)
at
org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:946)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1001)
at java.io.DataInputStream.read(DataInputStream.java:100)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:88)
at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:62)
at
org.apache.hadoop.hdfs.DFSTestUtil.readFileBuffer(DFSTestUtil.java:354)
at org.apache.hadoop.hdfs.DFSTestUtil.readFile(DFSTestUtil.java:346)
at
org.apache.hadoop.hdfs.TestClientProtocolForPipelineRecovery.testPipelineRecoveryWithTransferBlock(TestClientProtocolForPipelineRecovery.java:588)
{code}
Thanks.
> Add a unit test for HDFS-4660
> -----------------------------
>
> Key: HDFS-10652
> URL: https://issues.apache.org/jira/browse/HDFS-10652
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode, hdfs
> Reporter: Yongjun Zhang
> Assignee: Vinayakumar B
> Attachments: HDFS-10652-002.patch, HDFS-10652.001.patch,
> HDFS-10652.003.patch, HDFS-10652.004.patch, HDFS-10652.005.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
