[
https://issues.apache.org/jira/browse/HDFS-10460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15344331#comment-15344331
]
Kai Zheng commented on HDFS-10460:
----------------------------------
Thanks for the update, Rakesh. It looks much close now:
1. In StripedBlockChecksumReconstructor#reconstruct(), could we move the most
of the new codes to a new function like
{{checksumWithTargetOutput(targetBuffer)}}? Doing this would separate the
checksum logic from the reconstruction stuffs and when we add to handle more
failures than one, the new function can be reused.
2. Ref. the following test case: 1) The 1 requestedLen case is good to have,
might also have a larger one like 10? 2) Don't quite get why we have an array
for the only one filepath. 3) You have quite a few tests like this, guess you
could move the codes of preparing for the test file to a common place.
{code}
+ /**
+ * Test to verify that the checksum can be computed for a small file less
than
+ * bytesPerCRC size.
+ */
+ @Test(timeout = 90000)
+ public void testStripedFileChecksumWithMissedDataBlocksRangeQuery16()
+ throws Exception {
+ int fileLength = 100;
+ byte[] fileData = StripedFileTestUtil.generateBytes(fileLength);
+ String stripedFile3 = ecDir + "/stripedFileChecksum3";
+ String[] filePaths = new String[] {stripedFile3};
+
+ for (String filePath : filePaths) {
+ Path testPath = new Path(filePath);
+ DFSTestUtil.writeFile(fs, testPath, fileData);
+ }
+ testStripedFileChecksumWithMissedDataBlocksRangeQuery(stripedFile3, 1);
+ }
{code}
> Erasure Coding: Recompute block checksum for a particular range less than
> file size on the fly by reconstructing missed block
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-10460
> URL: https://issues.apache.org/jira/browse/HDFS-10460
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Rakesh R
> Assignee: Rakesh R
> Attachments: HDFS-10460-00.patch, HDFS-10460-01.patch,
> HDFS-10460-02.patch, HDFS-10460-03.patch
>
>
> This jira is HDFS-9833 follow-on task to address reconstructing block and
> then recalculating block checksum for a particular range query.
> For example,
> {code}
> // create a file 'stripedFile1' with fileSize = cellSize * numDataBlocks =
> 65536 * 6 = 393216
> FileChecksum stripedFileChecksum = getFileChecksum(stripedFile1, 10, true);
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
