[ https://issues.apache.org/jira/browse/YARN-8812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623324#comment-16623324 ]
Oleksandr Shevchenko commented on YARN-8812: -------------------------------------------- Changed generating symlink creating command by adding "--" flag to separate flags of the command and path parameters. The command looks like now: {code} ln -sf -- "/tmp/hadoop-yarn/nm-local-dir/usercache/yarn/filecache/49/testfile" "-symlink" {code} Could someone review the patch? Thanks! > Containers fail during creating a symlink which started with hyphen for a > resource file > --------------------------------------------------------------------------------------- > > Key: YARN-8812 > URL: https://issues.apache.org/jira/browse/YARN-8812 > Project: Hadoop YARN > Issue Type: Bug > Reporter: Oleksandr Shevchenko > Assignee: Oleksandr Shevchenko > Priority: Minor > Attachments: YARN-8812.001.patch > > > When we run some job and add a file with alias started with hyphen then a > container fails during creating a symlink for a resource file: > {noformat} > yarn jar hadoop-mapreduce-examples.jar pi -files testfile#-symlink 1 1 > {noformat} > or add a file to distributed cache in MR job by "job.addCacheFile" > Containers fail if resource file has a symlink started with hyphen with the > following error: > {noformat} > Stack trace: ExitCodeException exitCode=1: > /tmp/hadoop-yarn/nm-local-dir/usercache/yarn/appcache/application_1537449069809_0022/container_e01_1537449069809_0022_02_000001/launch_container.sh > ln: invalid option -- 'y' > Try 'ln --help' for more information. > at org.apache.hadoop.util.Shell.runCommand(Shell.java:572) > at org.apache.hadoop.util.Shell.run(Shell.java:466) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:768) > at > org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:306) > {noformat} > The main cause of the problem is "launch_container.sh" script whitch contain > the following command for creating a symlink: > {noformat} > ln -sf "/tmp/hadoop-yarn/nm-local-dir/usercache/yarn/filecache/49/testfile" > "-symlink" > {noformat} > As the result "-symlink" parse as "-s" flag but not as a symlink name. > The same job successfully passed when running on MRv1 but not on YARN since > symlinks create in different ways. Unix systems support names which start > with the hyphen. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org