[
https://issues.apache.org/jira/browse/BIGTOP-868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Blake Williams updated BIGTOP-868:
----------------------------------
Description:
I'm using a relatively clean installation of CentOS 6.3 minimal. I add the
Bigtop repo as per the instructions
[here|https://cwiki.apache.org/BIGTOP/how-to-install-hadoop-distribution-from-bigtop-050.html].
I have installed Hadoop common and Oozie using yum and I can run Java and
streaming map reduce jobs without any problems. Unfortunately, when I try to
run the Oozie examples that come bundled with Bigtop, I get a
java.lang.ClassNotFoundException
I can see from the HDFS audit logs that the oozie-examples-3.3.0.jar file gets
inspected, but never opened. These are the only four entries for the jar file
in the audit log for the time the workflow is running:
{noformat}
2013-03-12 14:42:07,394 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,399 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
<snip>
2013-03-12 14:42:07,547 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,550 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
{noformat}
The container logs I get from the webconsole on port 8088 show the following
exception, but offer no further clues:
{noformat}
2013-03-12 15:10:18,681 FATAL [IPC Server handler 2 on 57310]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363061307536_0002_m_000000_0 - exited : java.lang.RuntimeException:
Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:396)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class org.apache.oozie.example.SampleMapper
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1611)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:979)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1579)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1485)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1577)
... 17 more
{noformat}
I managed to grab the job.xml file out of the temp directory while the failing
stage of the workflow was running, and I can see that the jar file gets added
to the classpath property:
{noformat}
<property><name>mapreduce.job.classpath.files</name><value>/user/user/user-oozi/0000001-130312141058075-oozie-oozi-W/mr-node--map-reduce/map-reduce-launcher.jar,/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar</value><source>programatically</source></property>
{noformat}
... but the class is still apparently not found. I've set all debugging up to
DEBUG for all components and can find no more clues.
Have I simply misconfigured something, or is this actually a bug? I don't
really know what to do next.
I can attach full versions of these logs if necessary.
was:
I'm using a relatively clean installation of CentOS 6.3 minimal. I add the
Bigtop repo as per the instructions
[here|https://cwiki.apache.org/BIGTOP/how-to-install-hadoop-distribution-from-bigtop-050.html].
I have installed Hadoop common and Oozie using yum and I can run Java and
streaming map reduce jobs without any problems. Unfortunately, when I try to
run the Oozie examples that come bundled with Bigtop, I get a
java.lang.ClassNotFoundException
I can see from the HDFS audit logs that the oozie-examples-3.3.0.jar file gets
inspected, but never opened. These are the only four entries for the jar file
in the audit log for the time the workflow is running:
{noformat}
2013-03-12 14:42:07,394 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,399 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
<snip>
2013-03-12 14:42:07,547 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,550 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
{noformat}
The container logs I get from the webconsole on port 8088 show the following
exception, but offer no further clues:
{noformat}
2013-03-12 15:10:18,681 FATAL [IPC Server handler 2 on 57310]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363061307536_0002_m_000000_0 - exited : java.lang.RuntimeException:
Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:396)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class org.apache.oozie.example.SampleMapper
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1611)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:979)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1579)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1485)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1577)
... 17 more
{noformat}
I managed to grab the job.xml file out of the temp directory while the failing
stage of the workflow was running, and I can see that the jar file gets added
to the classpath property:
{noformat}
<property><name>mapreduce.job.classpath.files</name><value>/user/user/oozie-oozi/0000001-130312141058075-oozie-oozi-W/mr-node--map-reduce/map-reduce-launcher.jar,/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar</value><source>programatically</source></property>
{noformat}
... but the class is still apparently not found. I've set all debugging up to
DEBUG for all components and can find no more clues.
Have I simply misconfigured something, or is this actually a bug? I don't
really know what to do next.
I can attach full versions of these logs if necessary.
> Oozie examples fail with ClassNotFoundException
> -----------------------------------------------
>
> Key: BIGTOP-868
> URL: https://issues.apache.org/jira/browse/BIGTOP-868
> Project: Bigtop
> Issue Type: Bug
> Components: General
> Affects Versions: 0.5.0
> Environment: Bigtop 0.5.0, Sun Java 1.6.0_43, Centos 6.3 64-bit
> running in VirtualBox 4.2
> Reporter: Blake Williams
> Labels: oozie
>
> I'm using a relatively clean installation of CentOS 6.3 minimal. I add the
> Bigtop repo as per the instructions
> [here|https://cwiki.apache.org/BIGTOP/how-to-install-hadoop-distribution-from-bigtop-050.html].
> I have installed Hadoop common and Oozie using yum and I can run Java and
> streaming map reduce jobs without any problems. Unfortunately, when I try to
> run the Oozie examples that come bundled with Bigtop, I get a
> java.lang.ClassNotFoundException
> I can see from the HDFS audit logs that the oozie-examples-3.3.0.jar file
> gets inspected, but never opened. These are the only four entries for the jar
> file in the audit log for the time the workflow is running:
> {noformat}
> 2013-03-12 14:42:07,394 INFO FSNamesystem.audit: allowed=true ugi=user
> (auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12
> cmd=getfileinfo
> src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
> dst=null perm=null
> 2013-03-12 14:42:07,399 INFO FSNamesystem.audit: allowed=true ugi=user
> (auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12
> cmd=getfileinfo
> src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
> dst=null perm=null
> <snip>
> 2013-03-12 14:42:07,547 INFO FSNamesystem.audit: allowed=true ugi=user
> (auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12
> cmd=getfileinfo
> src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
> dst=null perm=null
> 2013-03-12 14:42:07,550 INFO FSNamesystem.audit: allowed=true ugi=user
> (auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12
> cmd=getfileinfo
> src=/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
> dst=null perm=null
> {noformat}
> The container logs I get from the webconsole on port 8088 show the following
> exception, but offer no further clues:
> {noformat}
> 2013-03-12 15:10:18,681 FATAL [IPC Server handler 2 on 57310]
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
> attempt_1363061307536_0002_m_000000_0 - exited : java.lang.RuntimeException:
> Error in configuring object
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
> at
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
> at
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:396)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
> ... 9 more
> Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class org.apache.oozie.example.SampleMapper
> not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1611)
> at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:979)
> at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
> ... 14 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> Class org.apache.oozie.example.SampleMapper not found
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1579)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
> ... 16 more
> Caused by: java.lang.ClassNotFoundException: Class
> org.apache.oozie.example.SampleMapper not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1485)
> at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1577)
> ... 17 more
> {noformat}
> I managed to grab the job.xml file out of the temp directory while the
> failing stage of the workflow was running, and I can see that the jar file
> gets added to the classpath property:
> {noformat}
> <property><name>mapreduce.job.classpath.files</name><value>/user/user/user-oozi/0000001-130312141058075-oozie-oozi-W/mr-node--map-reduce/map-reduce-launcher.jar,/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar</value><source>programatically</source></property>
> {noformat}
> ... but the class is still apparently not found. I've set all debugging up to
> DEBUG for all components and can find no more clues.
> Have I simply misconfigured something, or is this actually a bug? I don't
> really know what to do next.
> I can attach full versions of these logs if necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
