Blake Williams created BIGTOP-868:
-------------------------------------
Summary: Oozie examples fail with ClassNotFoundException
Key: BIGTOP-868
URL: https://issues.apache.org/jira/browse/BIGTOP-868
Project: Bigtop
Issue Type: Bug
Components: General
Affects Versions: 0.5.0
Environment: Bigtop 0.5.0, Sun Java 1.6.0_43, Centos 6.3 64-bit
running in VirtualBox 4.2
Reporter: Blake Williams
I'm using a relatively clean installation of CentOS 6.3 minimal. I add the
Bigtop repo as per the instructions
[here|https://cwiki.apache.org/BIGTOP/how-to-install-hadoop-distribution-from-bigtop-050.html].
I have installed Hadoop common and Oozie using yum and I can run Java and
streaming map reduce jobs without any problems. Unfortunately, when I try to
run the Oozie examples that come bundled with Bigtop, I get a
java.lang.ClassNotFoundException
I can see from the HDFS audit logs that the oozie-examples-3.3.0.jar file gets
inspected, but never opened. These are the only four entries for the jar file
in the audit log for the time the workflow is running:
{noformat}
2013-03-12 14:42:07,394 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,399 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
<snip>
2013-03-12 14:42:07,547 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
2013-03-12 14:42:07,550 INFO FSNamesystem.audit: allowed=true ugi=user
(auth:SIMPLE) via oozie (auth:SIMPLE) ip=/192.168.56.12 cmd=getfileinfo
src=/user/oozie/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar
dst=null perm=null
{noformat}
The container logs I get from the webconsole on port 8088 show the following
exception, but offer no further clues:
{noformat}
2013-03-12 15:10:18,681 FATAL [IPC Server handler 2 on 57310]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1363061307536_0002_m_000000_0 - exited : java.lang.RuntimeException:
Error in configuring object
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:106)
at
org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:72)
at
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:396)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:335)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1367)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:152)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:103)
... 9 more
Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: Class org.apache.oozie.example.SampleMapper
not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1611)
at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:979)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1579)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1603)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class
org.apache.oozie.example.SampleMapper not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1485)
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1577)
... 17 more
{noformat}
I managed to grab the job.xml file out of the temp directory while the failing
stage of the workflow was running, and I can see that the jar file gets added
to the classpath property:
{noformat}
<property><name>mapreduce.job.classpath.files</name><value>/user/user/oozie-oozi/0000001-130312141058075-oozie-oozi-W/mr-node--map-reduce/map-reduce-launcher.jar,/user/user/examples/apps/map-reduce/lib/oozie-examples-3.3.0.jar</value><source>programatically</source></property>
{noformat}
... but the class is still apparently not found. I've set all debugging up to
DEBUG for all components and can find no more clues.
Have I simply misconfigured something, or is this actually a bug? I don't
really know what to do next.
I can attach full versions of these logs if necessary.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
