Hi Terence, I'm quite new to Twill and not sure how to do that exactly. Could you show me how to modify the following example to do the same?
https://github.com/apache/incubator-twill/blob/master/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java On Fri, Jan 22, 2016 at 7:24 PM, Terence Yim <[email protected]> wrote: > Hi Kristoffer, > > Seems like the exception comes from the YARN class "ConverterUtils". I > believe need to start the application with the version 2.7.1 Hadoop > Jars. How to do start the twill application? Usually on a cluster with > hadoop installed, you can get all the hadoop jars in the classpath by > running this: > > export CP=`hadoop classpath` > java -cp .:$CP YourApp ... > > Assuming your app classes and Twill jars are in the current directory. > > Terence > > On Fri, Jan 22, 2016 at 4:54 AM, Kristoffer Sjögren <[email protected]> wrote: >> Here's the full stacktrace. >> >> Exception in thread "main" java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:497) >> at org.apache.twill.launcher.TwillLauncher.main(TwillLauncher.java:89) >> Caused by: java.lang.RuntimeException: >> java.lang.reflect.InvocationTargetException >> at com.google.common.base.Throwables.propagate(Throwables.java:160) >> at >> org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:61) >> at >> org.apache.twill.internal.appmaster.ApplicationMasterMain.main(ApplicationMasterMain.java:77) >> ... 5 more >> Caused by: java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) >> at >> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> at java.lang.reflect.Constructor.newInstance(Constructor.java:422) >> at >> org.apache.twill.internal.yarn.VersionDetectYarnAMClientFactory.create(VersionDetectYarnAMClientFactory.java:58) >> ... 6 more >> Caused by: java.lang.IllegalArgumentException: Invalid ContainerId: >> container_e25_1453466340022_0004_01_000001 >> at >> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:182) >> at >> org.apache.twill.internal.yarn.AbstractYarnAMClient.<init>(AbstractYarnAMClient.java:83) >> at >> org.apache.twill.internal.yarn.Hadoop21YarnAMClient.<init>(Hadoop21YarnAMClient.java:65) >> at >> org.apache.twill.internal.yarn.Hadoop22YarnAMClient.<init>(Hadoop22YarnAMClient.java:34) >> ... 11 more >> Caused by: java.lang.NumberFormatException: For input string: "e25" >> at >> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) >> at java.lang.Long.parseLong(Long.java:589) >> at java.lang.Long.parseLong(Long.java:631) >> at >> org.apache.hadoop.yarn.util.ConverterUtils.toApplicationAttemptId(ConverterUtils.java:137) >> at >> org.apache.hadoop.yarn.util.ConverterUtils.toContainerId(ConverterUtils.java:177) >> ... 14 more >> >> On Thu, Jan 21, 2016 at 10:59 PM, Kristoffer Sjögren <[email protected]> >> wrote: >>> Hi >>> >>> I'm trying the basic example [1] on yarn 2.7.1 but get an exception as >>> soon as the application starts on the resource manager that tells me >>> the container id cannot be parsed. >>> >>> java.lang.IllegalArgumentException: Invalid containerId: >>> container_e04_1427159778706_0002_01_000001 >>> >>> I don't have the exact stacktrace but I recall it failing in >>> ConverterUtils.toContainerId because it assumes that that the first >>> token is an application attempt to be parsed as an integer. This class >>> resides in hadoop-yarn-common 2.3.0. >>> >>> Is there any way to either tweak the container id or make twill use >>> the 2.7.1 jar instead? >>> >>> Cheers, >>> -Kristoffer >>> >>> >>> [1] >>> https://github.com/apache/incubator-twill/blob/master/twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java
