[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Target Version/s: 2.0.0-alpha, 1.1.0 (was: 1.1.0, 2.0.0-alpha) Status: Patch Available (was: Open) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 1.0.2, 0.23.1 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch, MR-3993_trunk.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: (was: MR-3993_branch1.patch) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch, MR-3993_trunk.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: MR-3993_trunk.patch MR-3993_branch1.patch Patches with Todd-suggested changes. > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch, MR-3993_branch1.patch, > MR-3993_trunk.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: MR-3993_branch1.patch The updated patch with following changes: * Added CompressionUtil for utility methods * Implemented CompressionUtil.wrappedRead() * Reduce (in-memory shuffle, and on-disk merge) use the wrapper instead of direct read * Test for the wrapper method > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: (was: MR-3993_branch1.patch) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: (was: MR-3993_branch1.patch) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: MR-3993_branch1.patch Updated patch for MR1. - With the codec being pluggable, we need to catch any possible errors/exceptions thrown by it. - Now, catching a Throwable as opposed to IOEx and RTEx. - Testing yet to be done. > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch, MR-3993_branch1.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-3993: Attachment: MR-3993_branch1.patch Uploading a preliminary patch for MR1. Need to add tests to ensure correct behavior. Any pointers on what might be the best way to force reading corrupt data in tests? > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon >Assignee: Karthik Kambatla > Attachments: MR-3993_branch1.patch > > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated MAPREDUCE-3993: -- Target Version/s: 1.1.0, 2.0.0 (was: 2.0.0, 1.0.3) No patches available in 1.0.3 timeframe. Please consider continuing contribution in 1.1 context. Thanks. > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-3993: --- Target Version/s: 1.0.3, 2.0.0 (was: 1.0.3, 0.23.3) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3993) reduce fetch catch clause should catch RTEs as well
[ https://issues.apache.org/jira/browse/MAPREDUCE-3993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Foley updated MAPREDUCE-3993: -- Target Version/s: 0.23.3, 1.0.3 (was: 0.23.3, 1.0.2) > reduce fetch catch clause should catch RTEs as well > --- > > Key: MAPREDUCE-3993 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3993 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1, mrv2 >Affects Versions: 0.23.1, 1.0.2 >Reporter: Todd Lipcon > > When using a compression codec for intermediate compression, some cases of > corrupt data can cause the codec to throw exceptions other than IOException > (eg java.lang.InternalError). This will currently cause the whole reduce task > to fail, instead of simply treating it like another case of a failed fetch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
