[ https://issues.apache.org/jira/browse/YARN-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Giovanni Matteo Fumarola updated YARN-9482: ------------------------------------------- Fix Version/s: 3.3.0 > DistributedShell job with localization fails in unsecure cluster > ---------------------------------------------------------------- > > Key: YARN-9482 > URL: https://issues.apache.org/jira/browse/YARN-9482 > Project: Hadoop YARN > Issue Type: Bug > Components: distributed-shell > Affects Versions: 3.3.0 > Reporter: Prabhu Joseph > Assignee: Prabhu Joseph > Priority: Major > Fix For: 3.3.0 > > Attachments: YARN-9482-001.patch, YARN-9482-002.patch, > YARN-9482-003.patch, YARN-9482-004.patch > > > DistributedShell job with localization fails in unsecure cluster. The client > localizes the input files to home directory (job user) whereas the AM runs as > yarn user reads from it's home directory. > *Command:* > {code} > yarn jar > /HADOOP/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0.jar > -shell_command ls -shell_args / -jar > /HADOOP/hadoop-3.2.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.2.0.jar > -localize_files /tmp/prabhu > {code} > {code} > Exception in thread "Thread-4" java.io.UncheckedIOException: Error during > localization setup > at > org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.lambda$run$0(ApplicationMaster.java:1495) > at > java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) > at > java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) > at > org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.run(ApplicationMaster.java:1481) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.io.FileNotFoundException: File does not exist: > hdfs://yarn-ats-1:8020/user/yarn/DistributedShell/application_1554817981283_0003/prabhu > at > org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1586) > at > org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1579) > at > org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) > at > org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1594) > at > org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster$LaunchContainerRunnable.lambda$run$0(ApplicationMaster.java:1487) > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org