[
https://issues.apache.org/jira/browse/HDFS-4960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13702787#comment-13702787
]
Colin Patrick McCabe commented on HDFS-4960:
--------------------------------------------
This change seems problematic. The .meta header exists so that we can check
the version of the block file. If we ignore it, we've effectively locked
ourselves into a single version forever.
I don't see how we can accept this change unless an alternate way of versioning
the block files is also proposed. One possible way would be through the naming
of the block files. But this is something we should discuss before throwing
out the existing mechanism. Also, if we're going to ignore the header for
no-checksum, we should ignore it in the checksum case as well.
Also, did you measure the performance impact?
> Unnecessary .meta seeks even when skip checksum is true
> -------------------------------------------------------
>
> Key: HDFS-4960
> URL: https://issues.apache.org/jira/browse/HDFS-4960
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Varun Sharma
> Assignee: Varun Sharma
> Attachments: 4960-branch2.patch, 4960-trunk.patch
>
>
> While attempting to benchmark an HBase + Hadoop 2.0 setup on SSDs, we found
> unnecessary seeks into .meta files, each seek was a 7 byte read at the head
> of the file - this attempts to validate the version #. Since the client is
> requesting no-checksum, we should not be needing to touch the .meta file at
> all.
> Since the purpose of skip checksum is to also avoid the performance penalty
> of the extra seek, we should not be seeking into .meta if skip checksum is
> true
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
