Re: \r\n problem in LineRecordReader.java

[Raghu Angadi]

Bwolen Yang wrote:
Here is probably the cause of this bug:
   public int read(byte b[], int off, int len) throws IOException {
     // make sure that it ends at a checksum boundary
     long curPos = getPos();
     long endPos = len+curPos/bytesPerSum*bytesPerSum;
     return readBuffer(b, off, (int)(endPos-curPos));
   }

Here, the caller calls the function with 127 bytes, and bytesPerSum is 256.

Is this from looking at the code or you actually saw the values like 
this at runtime?

I think 'len' is never supposed to be less than bytesPerChecksum because 
there is a BufferedInputStream between with a buffer size of 
io.buffer.size (default 4096). So this buffer size is supposed to be 
larger than bytesPerChecksum (requirement changes with HADOOP-1450 where 
it uses buffer size equal to bytesPerChecksum).

Raghu.