[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12926102#action_12926102 ] Hudson commented on MAPREDUCE-1686: --- Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/]) > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming, test >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Assignee: Paul Burkhardt >Priority: Minor > Fix For: 0.22.0 > > Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch, > HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893982#action_12893982 ] Amareshwari Sriramadasu commented on MAPREDUCE-1686: +1 Patch looks good. Test failure is because of MAPREDUCE-1834. Will check this in. > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Priority: Minor > Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch, > HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893981#action_12893981 ] Hadoop QA commented on MAPREDUCE-1686: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12450844/HADOOP-1686-2.patch against trunk revision 980316. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 7 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/console This message is automatically generated. > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Priority: Minor > Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch, > HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893511#action_12893511 ] Amareshwari Sriramadasu commented on MAPREDUCE-1686: bq. I ran both the ant test and ant test-patch targets with success because the testjar is created in the compile-core-test target. I need more information to understand your request. I still could not succeed running the newly added test. Are you testing your patch against branch 0.20 instead of trunk? because there is no such target as "compile-core-test" in trunk. It is compile-mapred-test and path for testjar is src/test/mapred/testjar in trunk. Please provide a patch for trunk. bq. The loadLibJar() and assert mentioned in your third comment is there to abort the unit test if the classpath is not properly loaded. Yes. I think the method and the assert associated with it, is unnecessary because StreamUtil.goodClassOrNull is doing the same thing : {code} clazz = conf.getClassByName(className); {code} > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Priority: Minor > Attachments: HADOOP-1686-1.patch, HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12893217#action_12893217 ] Paul Burkhardt commented on MAPREDUCE-1686: --- I attached a new patch to remove the printStackTrace() call in accordance to MAPREDUCE-571. Note that the pre-patched version will fail with a Java stack trace for ClassNotFoundException because it is thrown outside the method. I ran both the ant test and ant test-patch targets with success because the testjar is created in the compile-core-test target. I need more information to understand your request. The loadLibJar() and assert mentioned in your third comment is there to abort the unit test if the classpath is not properly loaded. Otherwise it may not be clear that the unit test failed because there was an issue in the configuration of the unit test. Let me know if you still want me to remove the assertion. Thank you for the comments. > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Priority: Minor > Attachments: HADOOP-1686-1.patch, HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12892952#action_12892952 ] Paul Burkhardt commented on MAPREDUCE-1686: --- Patch has been created and submitted. > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.1 >Reporter: Paul Burkhardt >Priority: Minor > Attachments: HADOOP-1686.patch > > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888358#action_12888358 ] Paul Burkhardt commented on MAPREDUCE-1686: --- Okay, I'll try and do that. Paul > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.2 >Reporter: Paul Burkhardt >Priority: Minor > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[ https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12888221#action_12888221 ] Amareshwari Sriramadasu commented on MAPREDUCE-1686: Paul, Can you create the patch with suggested change and a unit test, and upload here? > ClassNotFoundException for custom format classes provided in libjars > > > Key: MAPREDUCE-1686 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/streaming >Affects Versions: 0.20.2 >Reporter: Paul Burkhardt >Priority: Minor > > The StreamUtil::goodClassOrNull method assumes user-provided classes have > package names and if not, they are part of the Hadoop Streaming package. For > example, using custom InputFormat or OutputFormat classes without package > names will fail with a ClassNotFound exception which is not indicative given > the classes are provided in the libjars option. Admittedly, most Java > packages should have a package name so this should rarely come up. > Possible resolution options: > 1) modify the error message to include the actual classname that was > attempted in the goodClassOrNull method > 2) call the Configuration::getClassByName method first and if class not found > check for default package name and try the call again > {code} > public static Class goodClassOrNull(Configuration conf, String className, > String defaultPackage) { > Class clazz = null; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > if (clazz == null) { > if (className.indexOf('.') == -1 && defaultPackage != null) { > className = defaultPackage + "." + className; > try { > clazz = conf.getClassByName(className); > } catch (ClassNotFoundException cnf) { > } > } > } > return clazz; > } > {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.