[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926102#action_12926102
]
Hudson commented on MAPREDUCE-1686:
---
Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See
[https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/])
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming, test
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Assignee: Paul Burkhardt
Priority: Minor
Fix For: 0.22.0
Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch,
HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893981#action_12893981
]
Hadoop QA commented on MAPREDUCE-1686:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12450844/HADOOP-1686-2.patch
against trunk revision 980316.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 7 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac
compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
+1 core tests. The patch passed core unit tests.
-1 contrib tests. The patch failed contrib unit tests.
Test results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/testReport/
Findbugs warnings:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h6.grid.sp2.yahoo.net/600/console
This message is automatically generated.
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Priority: Minor
Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch,
HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893982#action_12893982
]
Amareshwari Sriramadasu commented on MAPREDUCE-1686:
+1
Patch looks good.
Test failure is because of MAPREDUCE-1834. Will check this in.
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Priority: Minor
Attachments: HADOOP-1686-1.patch, HADOOP-1686-2.patch,
HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893217#action_12893217
]
Paul Burkhardt commented on MAPREDUCE-1686:
---
I attached a new patch to remove the printStackTrace() call in accordance to
MAPREDUCE-571. Note that the pre-patched version will fail with a Java stack
trace for ClassNotFoundException because it is thrown outside the method.
I ran both the ant test and ant test-patch targets with success because the
testjar is created in the compile-core-test target. I need more information to
understand your request.
The loadLibJar() and assert mentioned in your third comment is there to abort
the unit test if the classpath is not properly loaded. Otherwise it may not be
clear that the unit test failed because there was an issue in the configuration
of the unit test. Let me know if you still want me to remove the assertion.
Thank you for the comments.
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Priority: Minor
Attachments: HADOOP-1686-1.patch, HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12893511#action_12893511
]
Amareshwari Sriramadasu commented on MAPREDUCE-1686:
bq. I ran both the ant test and ant test-patch targets with success because the
testjar is created in the compile-core-test target. I need more information to
understand your request.
I still could not succeed running the newly added test. Are you testing your
patch against branch 0.20 instead of trunk? because there is no such target as
compile-core-test in trunk. It is compile-mapred-test and path for testjar is
src/test/mapred/testjar in trunk. Please provide a patch for trunk.
bq. The loadLibJar() and assert mentioned in your third comment is there to
abort the unit test if the classpath is not properly loaded.
Yes. I think the method and the assert associated with it, is unnecessary
because StreamUtil.goodClassOrNull is doing the same thing :
{code}
clazz = conf.getClassByName(className);
{code}
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Priority: Minor
Attachments: HADOOP-1686-1.patch, HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12892952#action_12892952
]
Paul Burkhardt commented on MAPREDUCE-1686:
---
Patch has been created and submitted.
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.1
Reporter: Paul Burkhardt
Priority: Minor
Attachments: HADOOP-1686.patch
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12888221#action_12888221
]
Amareshwari Sriramadasu commented on MAPREDUCE-1686:
Paul, Can you create the patch with suggested change and a unit test, and
upload here?
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.2
Reporter: Paul Burkhardt
Priority: Minor
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1686) ClassNotFoundException for custom format classes provided in libjars
[
https://issues.apache.org/jira/browse/MAPREDUCE-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12888358#action_12888358
]
Paul Burkhardt commented on MAPREDUCE-1686:
---
Okay, I'll try and do that.
Paul
ClassNotFoundException for custom format classes provided in libjars
Key: MAPREDUCE-1686
URL: https://issues.apache.org/jira/browse/MAPREDUCE-1686
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming
Affects Versions: 0.20.2
Reporter: Paul Burkhardt
Priority: Minor
The StreamUtil::goodClassOrNull method assumes user-provided classes have
package names and if not, they are part of the Hadoop Streaming package. For
example, using custom InputFormat or OutputFormat classes without package
names will fail with a ClassNotFound exception which is not indicative given
the classes are provided in the libjars option. Admittedly, most Java
packages should have a package name so this should rarely come up.
Possible resolution options:
1) modify the error message to include the actual classname that was
attempted in the goodClassOrNull method
2) call the Configuration::getClassByName method first and if class not found
check for default package name and try the call again
{code}
public static Class goodClassOrNull(Configuration conf, String className,
String defaultPackage) {
Class clazz = null;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
if (clazz == null) {
if (className.indexOf('.') == -1 defaultPackage != null) {
className = defaultPackage + . + className;
try {
clazz = conf.getClassByName(className);
} catch (ClassNotFoundException cnf) {
}
}
}
return clazz;
}
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
