Hello, How can I verify the integrity of files copied to local from HDFS? Does HDFS store MD5s of full files anywhere? From what I can find, FileSystem.getFileChecksum() is relevant to replication and not comparison across filesystems (http://stackoverflow.com/questions/14563245/hdfs-file-checksum).
The Data Integrity section in HDFS Architecture (http://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsDesign.html) does not make it clear if, or how, copyToLocal verifies the integrity of the copied file. Thanks, Mike
