[
https://issues.apache.org/jira/browse/OOZIE-2944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048258#comment-16048258
]
Robert Kanter commented on OOZIE-2944:
--------------------------------------
{{LauncherAM}} should use {{Configuration}} instead of {{JobConf}}.
> Shell action example does not work with Oozie on Yarn on hadoop 2.6
> -------------------------------------------------------------------
>
> Key: OOZIE-2944
> URL: https://issues.apache.org/jira/browse/OOZIE-2944
> Project: Oozie
> Issue Type: Bug
> Reporter: Attila Sasvari
>
> I tried to execute example shell action with a pseudo hadoop 2.6.0. The
> workflow failed with the exception below :
> {code}
> $ ~/workspace/pseudo/hadoop-2.6.0/bin/yarn logs --applicationId
> application_1497357254962_0005
> 17/06/13 15:59:09 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 17/06/13 15:59:09 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Container: container_1497357254962_0005_01_000001 on 172.30.64.221_55437
> ==========================================================================
> LogType:stderr
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1397
> Log Contents:
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapred/Mapper
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.oozie.action.hadoop.LauncherAM.getMainArguments(LauncherAM.java:540)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:170)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:148)
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.Mapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 15 more
> LogType:stdout
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:2152
> Log Contents:
> Submitter user is: asasvari
> log4j: Trying to find [container-log4j.properties] using context classloader
> sun.misc.Launcher$AppClassLoader@6bc7c054.
> log4j: Using URL
> [jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties]
> for automatic log4j configuration.
> log4j: Reading configuration from URL
> jar:file:/Users/asasvari/workspace/pseudo/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar!/container-log4j.properties
> log4j: Hierarchy threshold set to [ALL].
> log4j: Parsing for [root] with value=[INFO,CLA, EventCounter].
> log4j: Level token is [INFO].
> log4j: Category root set to INFO
> log4j: Parsing appender named "CLA".
> log4j: Parsing layout options for "CLA".
> log4j: Setting property [conversionPattern] to [%d{ISO8601} %p [%t] %c: %m%n].
> log4j: End of parsing for "CLA".
> log4j: Setting property [containerLogDir] to
> [/Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_000001].
> log4j: Setting property [totalLogFileSize] to [0].
> log4j: setFile called:
> /Users/asasvari/workspace/pseudo/hadoop-2.6.0/logs/userlogs/application_1497357254962_0005/container_1497357254962_0005_01_000001/syslog,
> true
> log4j: setFile ended
> log4j: Parsed "CLA" options.
> log4j: Parsing appender named "EventCounter".
> log4j: Parsed "EventCounter" options.
> log4j: Finished configuring.
> Using login user for UGI
> Launcher AM configuration loaded
> Starting the execution of prepare actions
> Completed the execution of prepare actions successfully
> Failing Oozie Launcher, null
> Oozie Launcher, uploading action data to HDFS sequence file:
> hdfs://localhost:9000/user/asasvari/oozie-asas/0000002-170613150915408-oozie-asas-W/shell-node--shell/action-data.seq
> Stopping AM
> Callback notification attempts left 0
> Callback notification trying
> http://0.0.0.0:11000/oozie/callback?id=0000002-170613150915408-oozie-asas-W@shell-node&status=FAILED
> Callback notification to
> http://0.0.0.0:11000/oozie/callback?id=0000002-170613150915408-oozie-asas-W@shell-node&status=FAILED
> succeeded
> Callback notification succeeded
> LogType:syslog
> Log Upload Time:13-Jun-2017 15:58:45
> LogLength:1215
> Log Contents:
> 2017-06-13 15:58:43,105 WARN [main] org.apache.hadoop.util.NativeCodeLoader:
> Unable to load native-hadoop library for your platform... using builtin-java
> classes where applicable
> 2017-06-13 15:58:43,271 INFO [main] org.apache.hadoop.yarn.client.RMProxy:
> Connecting to ResourceManager at /0.0.0.0:8030
> 2017-06-13 15:58:43,844 INFO [main] org.apache.hadoop.io.compress.CodecPool:
> Got brand-new compressor [.deflate]
> 2017-06-13 15:58:43,972 INFO [main]
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl: Waiting for
> application to be successfully unregistered.
> 2017-06-13 15:58:44,078 INFO [AMRM Callback Handler Thread]
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl: Interrupted
> while waiting for queue
> java.lang.InterruptedException
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl$CallbackHandlerThread.run(AMRMClientAsyncImpl.java:274)
> {code}
> I believe this issue is related to
> [OOZIE-2918|https://issues.apache.org/jira/browse/OOZIE-2918].
> {{ShellActionExecutor}} extends {{JavaActionExecutor}} whose
> {{needToAddMapReduceToClassPath()}} returns {{false}}.
> {{LauncherMapper.getMainArguments()}} is called from {{LauncherAM}} via its
> {{run()}} method. {{LauncherMapper}} extends
> {{org.apache.hadoop.mapred.Mapper}} that is not on the classpath.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
