[
https://issues.apache.org/jira/browse/HADOOP-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suraj Nayak M updated HADOOP-10704:
-----------------------------------
Description:
A file is created using LocalFileSystem. When the file is corrupted/changed and
tried to read, it throws ChecksumException as the CRC does not match. But this
process does not move the affected file to "bad_files" directory.
Code :
public void testReportChecksumFailureOnCorruption() throws IOException {
String sampleData = "My Test Data. This will be corrupted soon";
FileSystem fs = LocalFileSystem.get(new Configuration());
Path rootPath = new Path(TEST_ROOT_DIR);
FSDataOutputStream fsOutStream = fs.create(new Path(rootPath
+ "/testFile.txt"));
ByteArrayInputStream inStream = new ByteArrayInputStream(
sampleData.getBytes());
IOUtils.copyBytes(inStream, fsOutStream, 4096, false);
fsOutStream.close();
inStream.close();
// Corrupt the file
File corruptFile = new File(TEST_ROOT_DIR + "/testFile.txt");
FileOutputStream fOutCorruptingStream = new FileOutputStream(
corruptFile, true);
fOutCorruptingStream.write(". This is bad data!".getBytes());
fOutCorruptingStream.close();
// Read the corrupted file using LocalFS
FileSystem readLFS = LocalFileSystem.get(new Configuration());
FSDataInputStream fsInStream = readLFS.open(new
Path(TEST_ROOT_DIR
+ "/testFile.txt"));
try {
byte[] b = new byte[10];
fsInStream.read(b, 0, 10);
} catch (Exception e) {
if (corruptFile.exists()) {
// Should not exit before moving the corrupt
file to bad_files
assertFalse(true);
} else {
// Should exit after moving the corrupt file to
bad_files
assertTrue(true);
}
}
fsInStream.close();
}
was:A file is created using LocalFileSystem. When the file is
corrupted/changed and tried to read, it throws ChecksumException as the CRC
does not match. But this process does not move the affected file to "bad_files"
directory.
> Corrupt Files are not moved to bad_files in LocalFileSystem
> -----------------------------------------------------------
>
> Key: HADOOP-10704
> URL: https://issues.apache.org/jira/browse/HADOOP-10704
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Reporter: Suraj Nayak M
> Priority: Minor
>
> A file is created using LocalFileSystem. When the file is corrupted/changed
> and tried to read, it throws ChecksumException as the CRC does not match. But
> this process does not move the affected file to "bad_files" directory.
> Code :
> public void testReportChecksumFailureOnCorruption() throws IOException {
> String sampleData = "My Test Data. This will be corrupted soon";
> FileSystem fs = LocalFileSystem.get(new Configuration());
> Path rootPath = new Path(TEST_ROOT_DIR);
> FSDataOutputStream fsOutStream = fs.create(new Path(rootPath
> + "/testFile.txt"));
> ByteArrayInputStream inStream = new ByteArrayInputStream(
> sampleData.getBytes());
> IOUtils.copyBytes(inStream, fsOutStream, 4096, false);
> fsOutStream.close();
> inStream.close();
> // Corrupt the file
> File corruptFile = new File(TEST_ROOT_DIR + "/testFile.txt");
> FileOutputStream fOutCorruptingStream = new FileOutputStream(
> corruptFile, true);
> fOutCorruptingStream.write(". This is bad data!".getBytes());
> fOutCorruptingStream.close();
> // Read the corrupted file using LocalFS
> FileSystem readLFS = LocalFileSystem.get(new Configuration());
> FSDataInputStream fsInStream = readLFS.open(new
> Path(TEST_ROOT_DIR
> + "/testFile.txt"));
> try {
> byte[] b = new byte[10];
> fsInStream.read(b, 0, 10);
> } catch (Exception e) {
> if (corruptFile.exists()) {
> // Should not exit before moving the corrupt
> file to bad_files
> assertFalse(true);
> } else {
> // Should exit after moving the corrupt file to
> bad_files
> assertTrue(true);
> }
> }
> fsInStream.close();
> }
--
This message was sent by Atlassian JIRA
(v6.2#6252)