[ https://issues.apache.org/jira/browse/HADOOP-11445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14257422#comment-14257422 ]
Ankit Kamboj commented on HADOOP-11445: --------------------------------------- The first -1 points to the patch itself rather than the testing results, so not sure which tests failed. Also, this patch is related to core components: hadoop-common and hadoop-mapreduce-client-core and unit tests for them have passed, then it should be fine right? Could somebody please take a look? > Bzip2Codec: Data block is skipped when position of newly created stream is > equal to start of split > -------------------------------------------------------------------------------------------------- > > Key: HADOOP-11445 > URL: https://issues.apache.org/jira/browse/HADOOP-11445 > Project: Hadoop Common > Issue Type: Bug > Affects Versions: 2.4.0 > Reporter: Ankit Kamboj > Attachments: HADOOP-11445.001.patch > > > bz2 input files are handled by FileInputFormat+LineRecordReader. In > LineRecordReader, bz2 specific compressed input stream is created to iterate > over records. After every new creation, the stream points to the beginning of > next data block. The logic to find the beginning of next block depends on > start of the split. The search begins at 10 bytes behind the start of split. > If the first search creates input stream whose position is before or at start > of split, next block beginning is sought (assuming that the record reader for > previous split would have already iterated over the the data block in which > current start of split lies). If the split start is just at the byte where a > newly created stream is positioned (start of data block), attempt is made to > find beginning of next data block. This doesn't seem correct because this > will result in jumping a whole block and will result in missing records. -- This message was sent by Atlassian JIRA (v6.3.4#6332)