Author: hairong
Date: Tue Aug 11 16:26:55 2009
New Revision: 803190
URL: http://svn.apache.org/viewvc?rev=803190&view=rev
Log:
HADOOP-6177. FSInputChecker.getPos() would return position greater than the
file size. Contributed by Hong Tang.
Modified:
hadoop/common/trunk/CHANGES.txt
hadoop/common/trunk/src/java/org/apache/hadoop/fs/FSInputChecker.java
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java
Modified: hadoop/common/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/hadoop/common/trunk/CHANGES.txt?rev=803190&r1=803189&r2=803190&view=diff
==============================================================================
--- hadoop/common/trunk/CHANGES.txt (original)
+++ hadoop/common/trunk/CHANGES.txt Tue Aug 11 16:26:55 2009
@@ -502,6 +502,9 @@
HADOOP-5638. More improvement on block placement performance. (hairong)
+ HADOOP-6180. NameNode slowed down when many files with same filename
+ were moved to Trash. (Boris Shkolnik via hairong)
+
BUG FIXES
HADOOP-5379. CBZip2InputStream to throw IOException on data crc error.
@@ -908,6 +911,9 @@
HADOOP-6124. Fix javac warning detection in test-patch.sh. (Giridharan
Kesavan via szetszwo)
+ HADOOP-6177. FSInputChecker.getPos() would return position greater
+ than the file size. (Hong Tang via hairong)
+
Release 0.20.1 - Unreleased
INCOMPATIBLE CHANGES
@@ -4466,9 +4472,6 @@
exponentially increasing number of records (up to 10,000
records/log). (Zheng Shao via omalley)
- HADOOP-6180. NameNode slowed down when many files with same filename
- were moved to Trash. (Boris Shkolnik via hairong)
-
BUG FIXES
HADOOP-2195. '-mkdir' behaviour is now closer to Linux shell in case of
Modified: hadoop/common/trunk/src/java/org/apache/hadoop/fs/FSInputChecker.java
URL:
http://svn.apache.org/viewvc/hadoop/common/trunk/src/java/org/apache/hadoop/fs/FSInputChecker.java?rev=803190&r1=803189&r2=803190&view=diff
==============================================================================
--- hadoop/common/trunk/src/java/org/apache/hadoop/fs/FSInputChecker.java
(original)
+++ hadoop/common/trunk/src/java/org/apache/hadoop/fs/FSInputChecker.java Tue
Aug 11 16:26:55 2009
@@ -174,6 +174,7 @@
assert(pos>=count);
// fill internal buffer
count = readChecksumChunk(buf, 0, buf.length);
+ if (count < 0) count = 0;
}
/*
Modified:
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java
URL:
http://svn.apache.org/viewvc/hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java?rev=803190&r1=803189&r2=803190&view=diff
==============================================================================
---
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java
(original)
+++
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestChecksumFileSystem.java
Tue Aug 11 16:26:55 2009
@@ -53,7 +53,11 @@
fout = localFs.create(testPath11);
fout.write("testing you".getBytes());
fout.close();
-
+
+ TestLocalFileSystem.readFile(localFs, testPath, 128);
+ TestLocalFileSystem.readFile(localFs, testPath, 512);
+ TestLocalFileSystem.readFile(localFs, testPath, 1024);
+
localFs.delete(localFs.getChecksumFile(testPath), true);
assertTrue("checksum deleted",
!localFs.exists(localFs.getChecksumFile(testPath)));
@@ -64,7 +68,7 @@
boolean errorRead = false;
try {
- TestLocalFileSystem.readFile(localFs, testPath);
+ TestLocalFileSystem.readFile(localFs, testPath, 1024);
}catch(ChecksumException ie) {
errorRead = true;
}
@@ -72,7 +76,7 @@
//now setting verify false, the read should succeed
localFs.setVerifyChecksum(false);
- String str = TestLocalFileSystem.readFile(localFs, testPath);
+ String str = TestLocalFileSystem.readFile(localFs, testPath, 1024);
assertTrue("read", "testing".equals(str));
}
Modified:
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java
URL:
http://svn.apache.org/viewvc/hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java?rev=803190&r1=803189&r2=803190&view=diff
==============================================================================
---
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java
(original)
+++
hadoop/common/trunk/src/test/core/org/apache/hadoop/fs/TestLocalFileSystem.java
Tue Aug 11 16:26:55 2009
@@ -35,13 +35,14 @@
stm.close();
}
- static String readFile(FileSystem fs, Path name) throws IOException {
- byte[] b = new byte[1024];
+ static String readFile(FileSystem fs, Path name, int buflen) throws
IOException {
+ byte[] b = new byte[buflen];
int offset = 0;
FSDataInputStream in = fs.open(name);
for(int remaining, n;
(remaining = b.length - offset) > 0 && (n = in.read(b, offset,
remaining)) != -1;
offset += n);
+ assertEquals(offset, Math.min(b.length, in.getPos()));
in.close();
String s = new String(b, 0, offset);
