[ https://issues.apache.org/jira/browse/HADOOP-6834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hong Tang updated HADOOP-6834: ------------------------------ Attachment: hadoop-6834-20100715.patch Straightforward patch with a new unit test. > TFile.append compares initial key against null lastKey > -------------------------------------------------------- > > Key: HADOOP-6834 > URL: https://issues.apache.org/jira/browse/HADOOP-6834 > Project: Hadoop Common > Issue Type: Bug > Components: io > Affects Versions: 0.20.1, 0.20.2 > Reporter: Ahad Rana > Assignee: Hong Tang > Attachments: hadoop-6834-20100715.patch > > > The following code in TFile.KeyReigster.close: > byte[] lastKey = lastKeyBufferOS.getBuffer(); > int lastLen = lastKeyBufferOS.size(); > if (tfileMeta.getComparator().compare(key, 0, len, lastKey, 0, > lastLen) < 0) { > throw new IOException("Keys are not added in sorted order"); > } > compares the initial key (passed in via TFile.Writer.append) against a > technically NULL lastKey. lastKey is not initialized until after the first > call to TFile.Writer.append. The underlying RawComparator interface used for > comparisons does not stipulate the proper behavior when either length 1 or > length 2 is zero. In the case of LongWritable, its WritableComparator > implementation does an unsafe read on the passed in byte arrays b1 and b2. > Since TFile pre-allocates the buffer used for storing lastKey, this passes a > valid buffer with zero count to LongWritable's comparator, which ignores > length and thus produces incorrect results. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.