[ https://issues.apache.org/jira/browse/HDFS-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13532724#comment-13532724 ]
Todd Lipcon commented on HDFS-3429: ----------------------------------- Hi Liang. I'm not sure if 0.94.2 has the code right to take advantage of this new feature quite yet -- given you see a bunch of the .meta files being read, it seems like it doesn't. So, that would explain why you don't see a performance difference. > DataNode reads checksums even if client does not need them > ---------------------------------------------------------- > > Key: HDFS-3429 > URL: https://issues.apache.org/jira/browse/HDFS-3429 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, performance > Affects Versions: 2.0.0-alpha > Reporter: Todd Lipcon > Assignee: Todd Lipcon > Attachments: hdfs-3429-0.20.2.patch, hdfs-3429-0.20.2.patch, > hdfs-3429.txt, hdfs-3429.txt, hdfs-3429.txt > > > Currently, even if the client does not want to verify checksums, the datanode > reads them anyway and sends them over the wire. This means that performance > improvements like HBase's application-level checksums don't have much benefit > when reading through the datanode, since the DN is still causing seeks into > the checksum file. > (Credit goes to Dhruba for discovering this - filing on his behalf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira