[ http://issues.apache.org/jira/browse/HADOOP-746?page=comments#action_12455248 ] dhruba borthakur commented on HADOOP-746: -----------------------------------------
Please allow me to see if I can sum up the ideas and try to arrive at a consensus: 1. There is a CRC that is computed and used internally by HDFS. HDFS computes this CRC per each block of the file. No files with *.crc are created in the namespace. 2. HDFS exposes an API that allows an application to retrieve the CRC of each block of an HDFS file. 3. HDFS exposes another API that an application can use to switch off CRC usage and calculation by HDFS. > CRC computation and reading should move into a nested FileSystem > ---------------------------------------------------------------- > > Key: HADOOP-746 > URL: http://issues.apache.org/jira/browse/HADOOP-746 > Project: Hadoop > Issue Type: Improvement > Components: fs > Affects Versions: 0.8.0 > Reporter: Owen O'Malley > Assigned To: Owen O'Malley > > Currently FileSystem provides both an interface and a mechanism for computing > and checking crc files. I propose splitting the crc code into a nestable > FileSystem that like the PhasedFileSystem has a backing FileSystem. Once the > Paths are converted to URI, this is fairly natural to express. To use crc > files, your uris will look like: > crc://hdfs:%2f%2fhost1:8020/ which is a crc FileSystem with an underlying > file system of hdfs://host1:8020 > This will allow users to use crc files where they make sense for their > application/cluster and get rid of the "raw" methods. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
