Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "WritingYarnApps" page has been changed by ChrisRiccomini: http://wiki.apache.org/hadoop/WritingYarnApps?action=diff&rev1=5&rev2=6 '''How can I distribute my application's jars distributed to all of the nodes in the YARN cluster that need it?''' + You can use the LocalResource to add resources to your application request. This will cause YARN to distribute the resource to the application master node, and all of the the container nodes. If the resource is a tgz, zip, or jar, you can have YARN unzip it. Then, all you need to do is add the unzipped folder to your classpath. For example, when creating your application request: + + {{{ val packageFile = new File(packagePath); + val packageUrl = ConverterUtils.getYarnUrlFromPath(FileContext.getFileContext.makeQualified(new Path(packagePath))); + + packageResource.setResource(packageUrl); + packageResource.setSize(packageFile.length()); + packageResource.setTimestamp(packageFile.lastModified()); + packageResource.setType(LocalResourceType.ARCHIVE); + packageResource.setVisibility(LocalResourceVisibility.APPLICATION); + + resource.setMemory(memory) + containerCtx.setResource(resource) + containerCtx.setCommands(ImmutableList.of(new ApplicationMasterExecutor(packagePath) + .addCommand("java -cp './package/*' some.class.to.Run " + + "1>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stdout " + + "2>" + ApplicationConstants.LOG_DIR_EXPANSION_VAR + "/stderr") + .execute(new Configuration)) + containerCtx.setLocalResources(Collections.singletonMap("package", packageResource)) + appCtx.setApplicationId(appId) + appCtx.setUser(user.getShortUserName) + appCtx.setAMContainerSpec(containerCtx) + request.setApplicationSubmissionContext(appCtx) + applicationsManager.submitApplication(request)}}} + + As you can see, the setLocalResources command takes a map of names to resources. The name becomes a sym link in your application's working directory, so you can just refer to the artifacts inside by using ./package/*. ''Note: Java's classpath (cp) argument is VERY sensitive. Make sure you get the syntax EXACTLY correct.'' + '''Why does my application's client have to send the app ID/attempt ID/cluster timestamp to my application master?''' This is actually going to be improved. It will likely become environment variables, or substituted variables (like the fail count). For more details, track this ticket: https://issues.apache.org/jira/browse/MAPREDUCE-3055
