[
https://issues.apache.org/jira/browse/MAPREDUCE-5160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13634285#comment-13634285
]
Zhijie Shen commented on MAPREDUCE-5160:
----------------------------------------
The exception observed when running aggregatewordhist:
{code}
2013-04-17 11:04:13,471 FATAL [IPC Server handler 7 on 61283]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1366220242098_0004_m_000007_0 - exited : java.lang.RuntimeException:
Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:425)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1489)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 17 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordHistogram$AggregateWordHistogramPlugin
at
org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:63)
at
org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createAggregator(UserDefinedValueAggregatorDescriptor.java:70)
at
org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.<init>(UserDefinedValueAggregatorDescriptor.java:84)
at
org.apache.hadoop.mapred.lib.aggregate.UserDefinedValueAggregatorDescriptor.<init>(UserDefinedValueAggregatorDescriptor.java:55)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getValueAggregatorDescriptor(ValueAggregatorJobBase.java:58)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.getAggregatorDescriptors(ValueAggregatorJobBase.java:69)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.initializeMySpec(ValueAggregatorJobBase.java:78)
at
org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJobBase.configure(ValueAggregatorJobBase.java:46)
... 22 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.examples.AggregateWordHistogram$AggregateWordHistogramPlugin
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:249)
at
org.apache.hadoop.mapreduce.lib.aggregate.UserDefinedValueAggregatorDescriptor.createInstance(UserDefinedValueAggregatorDescriptor.java:58)
... 29 more
{code}
> Aggregatewordcount and aggregatewordhist in hadoop-1 examples can not find
> their inner classes when running on Yarn
> -------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5160
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5160
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Reporter: Zhijie Shen
> Assignee: Zhijie Shen
>
> Aggregatewordcount and Aggregatewordhist of hadoop-1 cannot run on hadoop-2
> due to
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorJob.createValueAggregatorJob(String
> args[], Class<?> caller) is not available on hadoop-2 (see MAPREDUCE-5159).
> After I changed Aggregatewordcount and Aggregatewordhist to use
> createValueAggregatorJob(String args[], Class<? extends
> ValueAggregatorDescriptor>[] descriptors), which is available on hadoop-2,
> the two examples could be accepted and run on Yarn.
> However, the two examples still failed, because their inner classes,
> WordCountPlugInClass and AggregateWordHistogramPlugin, cannot be found in
> runtime, respectively. Both the plugin classes extend
> org.apache.hadoop.mapred.lib.aggregate.ValueAggregatorBaseDescriptor.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
