ClassNotFoundException for custom format classes provided in libjars --------------------------------------------------------------------
Key: MAPREDUCE-1686 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/streaming Affects Versions: 0.20.2 Reporter: Paul Burkhardt Priority: Trivial The StreamUtil::goodClassOrNull method assumes user-provided classes have package names and if not, they are part of the Hadoop Streaming package. For example, using custom InputFormat or OutputFormat classes without package names will fail with a ClassNotFound exception which is not indicative given the classes are provided in the libjars option. Admittedly, most Java packages should have a package name so this should rarely come up. Possible resolution options: 1) modify the error message to include the actual classname that was attempted in the goodClassOrNull method 2) call the Configuration::getClassByName method first and if class not found check for default package name and try the call again {code} public static Class goodClassOrNull(Configuration conf, String className, String defaultPackage) { Class clazz = null; try { clazz = conf.getClassByName(className); } catch (ClassNotFoundException cnf) { } if (clazz == null) { if (className.indexOf('.') == -1 && defaultPackage != null) { className = defaultPackage + "." + className; try { clazz = conf.getClassByName(className); } catch (ClassNotFoundException cnf) { } } } return clazz; } {code} -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.