I don't think it is an issue with classpath on the client - since it gets to start AppMaster container
Is any other YARN app runs OK? May be YarnConfiguration.YARN_APPLICATION_CLASSPATH is not producing right jars Look at HelloWorld code for yarnClasspath On Mon, Jun 12, 2017 at 1:22 PM, Chris Hebert < chris.hebert-...@digitalreasoning.com> wrote: > I hate to ask this here, but it won't work, so whatever. > > I followed the HelloWorld section of the Getting Started guide < > http://twill.apache.org/GettingStarted.html> on my cluster with Hadoop and > Zookeeper set up and functioning properly. > > git clone https://github.com/apache/twill.git > cd twill > mvn clean install -DskipTests > > export > CP=twill-examples/yarn/target/twill-examples-yarn-0.12.0- > SNAPSHOT.jar:`hadoop > classpath` > java -cp $CP org.apache.twill.example.yarn.HelloWorld > my.zookeeper.domain:2181 > > Yes, `hadoop classpath` echoes all the relevant jar directories. > > The command runs well for a bit with multiple: > [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn application > status for HelloWorldRunnable application_1111111111111_0001: ACCEPTED > > until: > [ STARTING] DEBUG o.a.hadoop.service.AbstractService - Service: > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl entered state > STOPPED > [ STARTING] DEBUG org.apache.hadoop.ipc.Client - stopping client from > cache: org.apache.hadoop.ipc.Client@4d465b11 > [ STARTING] DEBUG o.a.twill.yarn.YarnTwillController - Yarn application > status for HelloWorldRunnable application_1111111111111_0001: FAILED > ... > java.util.concurrent.ExecutionException: java.lang.RuntimeException: Yarn > application completed with failure HelloWorldRunnable... > > The ResourceManager reveals: > Application application_1111111111111_0001 failed 2 times due to AM > Container for appattempt_1111111111111_0001_000002 exited with exitCode: 1 > ... > Diagnostics: Exception from container-launch. > > The corresponding YARN logs for each DataNode reveal: > Exception in thread "main" java.lang.NoClassDefFoundError: > org/apache/hadoop/conf/Configuration > at java.lang.Class.getDeclaredMethods0(Native Method) > at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) > at java.lang.Class.privateGetMethodRecursive(Class.java:3048) > at java.lang.Class.getMethod0(Class.java:3018) > at java.lang.Class.getMethod(Class.java:1784) > at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:70) > Caused by: java.lang.ClassNotFoundException: > org.apache.hadoop.conf.Configuration > at java.net.URLClassLoader.findClass(URLClassLoader.java:381) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 6 more > Launch class (org.apache.twill.internal.appmaster.ApplicationMasterMain) > using classloader java.net.URLClassLoader with classpath: [ > *A list of several classpaths like > "file:/some/path/yarn/local/usercache/my.username/appcache/application_ > 1111111111111_0001/container_e10_ > 1111111111111 > _0001_02_000001/application.jar/lib/twill-examples-yarn-0. > 12.0-SNAPSHOT.jar" > But none of which are paths to any Hadoop jars of the sort that are > referenced in $CP* > ] > > What am I missing? > > I've spent an embarrassingly large amount of time on this fiddling with > environment variables and Hadoop configuration. (I'm an intern learning > this stuff the hard way, so it's not really embarrassing, just > substantial.) >
