[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-1927: - Fix Version/s: 0.23.0 > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.23.0 > > Attachments: MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v13.trunk-hadoop-mapreduce.patch.txt, > MR-1927.v9.trunk-hadoop-mapreduce.patch, > construct-all-gz-header-types-20100812.zip > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tom White updated MAPREDUCE-1927: - Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) +1 I just committed this. Thanks Greg. > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Attachments: MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v13.trunk-hadoop-mapreduce.patch.txt, > MR-1927.v9.trunk-hadoop-mapreduce.patch, > construct-all-gz-header-types-20100812.zip > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Status: Patch Available (was: Open) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v13.trunk-hadoop-mapreduce.patch.txt, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Attachment: MR-1927.v13.trunk-hadoop-mapreduce.patch.txt Fixed the internal structure of the extra field in one of the test files (not relevant to the test, but a matter of spec correctness), as well as the corresponding file length in the unit test (3409 -> 3413 bytes). No substantive changes since v12. > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v13.trunk-hadoop-mapreduce.patch.txt, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Status: Open (was: Patch Available) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Status: Patch Available (was: Open) Extended hadoop-mapreduce unit test for hadoop-common's gzip and bzip2 codecs. (Supplements TestCodec in hadoop-common.) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Attachment: MR-1927.v12.trunk-hadoop-mapreduce.patch Updated (final?) patch. This corrects the internals of the gzip extra field in one of the test files (not relevant to the test, but matter of spec-compliance), changes a native-libs-only assertion to a warning to work around Apache Hudson's lack of support, and removes all but three commented-out code blocks. The three remaining blocks are all a result of other bugs: the aforementioned Hudson native-library one (lack of coverage of a key infrastructure component is a problem, and it's getting worse with every new native codec); HADOOP-6799 (broken reset() on gzip compression); and HADOOP-6852 (broken bzip2 decompression, apparently). I realize commented-out code is frowned upon as a style issue, but I would rather not remove useful tests that are currently blocked by things outside my (immediate) control. I'll defer to committer's opinion, however. This passed "ant test-patch" on a 10-node cluster, so I'm going to "submit patch" to see if it gets past Apache's Hudson, too. > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v12.trunk-hadoop-mapreduce.patch, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Attachment: construct-all-gz-header-types-20100812.zip Script and gzip component subfiles to generate all 32 unencrypted flagbits variants (that is, text/binary, comment/no comment, header CRC/no header CRC, etc.). These are not currently used by the unit test, but they could be. Is it worth it? (Note that gzip versions up through 1.4 cannot be used to test the 16 header-CRC variants. A fix was checked into gzip's trunk on 20100703, however: http://git.savannah.gnu.org/cgit/gzip.git/commit/?id=97e225c0e8ee24f3fc9ff7c5cec63e74aefbb6a0 . zlib-based gzip decoders [zlib >= 1.2.1.2] can read all 32 variants successfully.) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: construct-all-gz-header-types-20100812.zip, > MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Status: Open (was: Patch Available) Updated, test-patch-passing patch coming shortly. > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Status: Patch Available (was: Open) > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (MAPREDUCE-1927) unit test for HADOOP-6835 (concatenated gzip support)
[ https://issues.apache.org/jira/browse/MAPREDUCE-1927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Roelofs updated MAPREDUCE-1927: Attachment: MR-1927.v9.trunk-hadoop-mapreduce.patch Test code, six associated test files, and build.xml changes to copy them to the appropriate location. > unit test for HADOOP-6835 (concatenated gzip support) > - > > Key: MAPREDUCE-1927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1927 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Affects Versions: 0.22.0 >Reporter: Greg Roelofs >Assignee: Greg Roelofs >Priority: Minor > Fix For: 0.22.0 > > Attachments: MR-1927.v9.trunk-hadoop-mapreduce.patch > > > More extensive test of concatenated gzip (and bzip2) decoding support for > HADOOP-6835 (and HADOOP-4012 and HADOOP-6852). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
