Hi Aakash, Thanks for the finding. I've created a JIRA for investigating the problem.
https://issues.apache.org/jira/browse/TWILL-65 Terence On Sun, Apr 6, 2014 at 5:00 PM, aakash aakash <[email protected]> wrote: > Thanks Alvin!! > > I tried the solution, you suggested but application is throwing exception : > > 14/04/06 14:57:10 INFO internal.ServiceMain: Service > org.apache.twill.internal.appmaster.ApplicationMasterService@3c9c31 > started. > 14/04/06 14:57:10 INFO controller.ControllerEpochListener: > [ControllerEpochListener on 1]: Initialized controller epoch to 1 and > zk version 0 > 14/04/06 14:57:10 INFO appmaster.ApplicationMasterService: Request 1 > container with capability <memory:512, vCores:1> > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Got > container container_1396815710754_0007_01_000002 > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Starting > runnable BundledJarRunnable with > RunnableProcessLauncher{container=org.apache.twill.internal.yarn.Hadoop21YarnContainerInfo@1f30035} > 14/04/06 14:57:12 INFO appmaster.RunnableProcessLauncher: Launching in > container container_1396815710754_0007_01_000002 at > localhost/127.0.0.1, [java -Djava.io.tmpdir=tmp > -Dyarn.container=$YARN_CONTAINER_ID > -Dtwill.runnable=$TWILL_APP_NAME.$TWILL_RUNNABLE_NAME -cp > launcher.jar:$HADOOP_CONF_DIR -Xmx359m > org.apache.twill.launcher.TwillLauncher container.jar > org.apache.twill.internal.container.TwillContainerMain true > 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr] > 14/04/06 14:57:12 INFO impl.ContainerManagementProtocolProxy: Opening > proxy : localhost:55102 > 14/04/06 14:57:12 INFO appmaster.ApplicationMasterService: Runnable > BundledJarRunnable fully provisioned with 1 instances. > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: Container > container_1396815710754_0007_01_000002 completed with > COMPLETE:Exception from container-launch: > org.apache.hadoop.util.Shell$ExitCodeException: > at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) > at org.apache.hadoop.util.Shell.run(Shell.java:379) > at > org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) > at > org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) > at > org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) > at java.util.concurrent.FutureTask.run(FutureTask.java:138) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) > at java.lang.Thread.run(Thread.java:662) > > > . > 14/04/06 14:57:18 WARN appmaster.RunningContainers: Container > container_1396815710754_0007_01_000002 exited abnormally with state > COMPLETE, exit code 10. > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: All > containers completed. Shutting down application master. > 14/04/06 14:57:18 INFO appmaster.ApplicationMasterService: Stop > application master with spec: > {"name":"ExampleBundledJarApp","runnables":{"BundledJarRunnable":{"name":"BundledJarRunnable","runnable":{"classname":"org.apache.twill.ext.BundledJarRunnable","name":"BundledJarRunnable","arguments":{}},"resources":{"cores":1,"memorySize":512,"instances":1,"uplink":-1,"downlink":-1,"hosts":[],"racks":[]},"files":[{"name":"job-web-service-1.0.jar","uri":"file:/twill/ExampleBundledJarApp/03f7c49a-6471-40b6-9992-af07905d44cc/job-web-service-1.0.jar.7e6aaa7d-fc66-462d-abeb-3b2fb3b7d812.jar","lastModified":1396821425000,"size":75347840,"archive":false,"pattern":null}]}},"orders":[{"names":["BundledJarRunnable"],"type":"STARTED"}],"handler":{"classname":"org.apache.twill.internal.LogOnlyEventHandler","configs":{}}} > 14/04/06 14:57:18 INFO appmaster.RunningContainers: Stopping all > instances of BundledJarRunnable > > > > > > Twill example code: > > ------------------- > > > > //Example Application > > > private static class ExampleBundledJarApp implements TwillApplication { > 77 private final String jarName; > 78 private final URI jarURI; > 79 > 80 public ExampleBundledJarApp(String jarName, URI jarURI) { > 81 this.jarName = jarName; > 82 this.jarURI = jarURI; > 83 } > 84 > 85 @Override > 86 public TwillSpecification configure() { > 87 return TwillSpecification.Builder.with() > 88 > .setName("ExampleBundedJarApp").withRunnable() > 89 .add("BundledJarRunnable", > new BundledJarRunnable()) > 90 > .withLocalFiles().add(jarName, jarURI, false).apply() > 91 .anyOrder().build(); > 92 } > 93 } > > > > > > //calling twill runner > > Arguments arguments = new Arguments.Builder() > 204 > .setJarFileName("lib/job-web-service-1.0.jar").setLibFolder("lib") > 205 > .setMainClassName(Main.class.getPackage().getName()).setMainArgs(new > String[]{"1"}) > 206 .createArguments(); > 207 > 208 > 209 > 210 LOG.info("arguments jar file:: "+ > arguments.getJarFileName()); > 211 LOG.info("arguments lib folder:: "+ > arguments.getLibFolder()); > 212 LOG.info("arguments main class:: "+ > arguments.getMainClassName()); > 213 LOG.info("arguments main args:: "+ > arguments.getMainArgs()); > 214 > 215 LOG.info("arguments:: "+ > Arrays.asList(arguments.toArray())); > 216 > 217 > 218 File jarFile = new File(arguments.getJarFileName()); > 219 Preconditions.checkArgument(jarFile != null); > 220 Preconditions.checkState(jarFile.exists(), "jar file > does not exist: "+jarFile.getAbsolutePath()); > 221 Preconditions.checkState(jarFile.canRead()); > 222 > 223 > 224 TwillPreparer twillPreparer = runnerService > 225 .prepare(new > ExampleBundledJarApp(jarFile.getName(), jarFile.toURI())) > 226 .withArguments("BundledJarRunnable", > arguments.toArray()) > 227 .addLogHandler( > 228 new > PrinterLogHandler(new PrintWriter(System.out))); > 229 > 230 twillPreparer.start(); > > > There are two problem I observed here: > > 1. if u dont set any main arg it throw an exception, need to be fixed by > BundledJarRunnable.Arguments by checking main args for being null > 2. I see arguments are not populated in the json (check the container error > log above). > > > Thanks, > Aakash > > > > > > > On Tue, Apr 1, 2014 at 4:35 PM, Alvin Wang <[email protected]> wrote: > >> I recently committed something that can serve as a workaround for this, >> when I was trying to get Presto running with Twill. Presto was using Guava >> 16 which was incompatible with Twill's Guava 13. >> >> You can take a look at BundledJarRunnable available in the 0.2.0-incubating >> release (sample >> usage< >> https://git-wip-us.apache.org/repos/asf?p=incubator-twill.git;a=blob;f=twill-examples/yarn/src/main/java/org/apache/twill/example/yarn/BundledJarExample.java;h=3cc88a1b7624270c2d870c8a249185d608b09613;hb=d4a352de4a62abdf4607af77d610597630e0ee1a >> >). >> It's currently very limited in terms of functionality, but you can use it >> to run an application without having to worry about Twill's classpath. You >> can specify a bundle jar file, folder containing the dependencies within >> the jar file, main class name, and arguments to send to the main method. >> >> The BundledJarRunnable will load the jars in the folder containing the >> dependencies and also the bundle jar into a clean classloader, so it won't >> conflict with Twill's dependencies. To use this, do the following: >> >> 1. Package your program into a bundle jar. I used the following in my >> pom.xml: >> >> <build> >> <plugins> >> <plugin> >> <groupId>org.apache.felix</groupId> >> <artifactId>maven-bundle-plugin</artifactId> >> <version>2.3.7</version> >> <extensions>true</extensions> >> <configuration> >> <instructions> >> <Embed-Dependency>*;inline=false</Embed-Dependency> >> <Embed-Transitive>true</Embed-Transitive> >> <Embed-Directory>lib</Embed-Directory> >> </instructions> >> </configuration> >> </plugin> >> </plugins> >> </build> >> >> 2. Configure your TwillApplication to include a BundledJarRunnable() and >> the jar file: >> ...withRunnable().add("Test123", new >> BundledJarRunnable()).withLocalFiles().add("sdf.jar", localJarURI, false) >> >> 3. Configure your TwillPreparer to pass runtime arguments (main class name, >> main method args, etc.) to your runnable: >> >> BundledJarRunner.Arguments arguments = new >> BundledJarRunner.Arguments.Builder() >> .set ... >> ... >> .build(); >> >> final TwillController controller = twillRunner.prepare( >> new ExampleBundledJarApp() >> .withArguments("Test123", arguments.toArray()) >> .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out, >> true))) >> .start(); >> >> -- >> >> Let me know if you have any questions. >> >> Alvin >> >> >> On Tue, Apr 1, 2014 at 3:01 AM, aakash aakash <[email protected] >> >wrote: >> >> > Hi, >> > >> > I am trying to start a rest web service on hadoop using Twill, but >> > unfortunately jersey use asm 3.1 but in Twill we use asm 4 and source is >> > also different. Which is now a big problem for me. >> > >> > Is there any workaround for this. Any help will be highly appreciated. >> > >> > Thanks, >> > Aakash >> > >>
