[ http://issues.apache.org/jira/browse/HADOOP-738?page=comments#action_12451750 ] Doug Cutting commented on HADOOP-738: -------------------------------------
> Let's add a command line option to turn CRC export on. As I said, I'm fine with that, but that sounds like a separate issue than the one described: It doesn't fix this issue, but rather hides it for some users. When crc's *are* enabled, it shouldn't cause problems to 'dfs -put' files that have crcs. That's the issue here. Disabling crcs for 'dfs -get' for folks who have ECC memory and don't ever have file corruption issues on local disks is not something I object to. > This will let sophisticated users who have a need for CRCs use them [ ... ] So 'sophisticated' now means that you don't have ECC memory and reliable hard drives? > dfs get or copyToLocal should not copy crc file > ----------------------------------------------- > > Key: HADOOP-738 > URL: http://issues.apache.org/jira/browse/HADOOP-738 > Project: Hadoop > Issue Type: Bug > Components: dfs > Affects Versions: 0.8.0 > Environment: all > Reporter: Milind Bhandarkar > Assigned To: Milind Bhandarkar > Fix For: 0.9.0 > > Attachments: hadoop-crc.patch > > > Currently, when we -get or -copyToLocal a directory from DFS, all the files > including crc files are also copied. When we -put or -copyFromLocal again, > since the crc files already exist on DFS, this put fails. The solution is not > to copy checksum files when copying to local. Patch is forthcoming. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
