[
https://issues.apache.org/jira/browse/HADOOP-1259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12488743
]
Raghu Angadi commented on HADOOP-1259:
--------------------------------------
There are two types of checksums :
1) File Level checksum in ChecksumFileSystem.
2) Block Level Checksum in DFS ( HADOOP-1134).
This issue/requirement is only about DFS support for (1).
> So, if we decide to support this (arbitrary block sizes and bytesPerChecksum
> in HDFS) then I suggest we implement it
> this way: the final checksum in each block is permitted to be for fewer bytes
> than bytesPerChecksum.
Post HADOOP-1134 , this will be case. i.e. last checksum could be for less than
bytesPerChecksum since checksum is always. Though it would mostly be better to
enforce the condition here too.
Fixing this jira would we simple and might reduce further propagation of this
mismatch.
> DFS should enforce block size is a multiple of io.bytes.per.checksum
> ---------------------------------------------------------------------
>
> Key: HADOOP-1259
> URL: https://issues.apache.org/jira/browse/HADOOP-1259
> Project: Hadoop
> Issue Type: Improvement
> Reporter: Raghu Angadi
>
> DFSClient currently does not enforce that dfs.block.size is a multiple
> io.bytes.per.checksum. This not really problem currently but can future
> upgrades like HADOOP-1134 (see one of the comments
> http://issues.apache.org/jira/browse/HADOOP-1134#action_12488542 there).
> I propose DFSClient should fail loudly and ask the user politely to change
> the config to meet this conidtion. Of course we will change the documentation
> for dfs.block.size also.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
