[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-3123: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Hitesh! Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3123: --- Status: Patch Available (was: Open) Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container:
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3123: --- Attachment: MR-3123.1.patch Attaching a simple patch for now which addresses the immediate bug related to symlinks with special chars. Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-3123: - Status: Open (was: Patch Available) you forgot to add the apache header to the new file. Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2011-09-27 20:58:48,951 INFO
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3123: --- Attachment: MR-3123.2.patch Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch, MR-3123.2.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2011-09-27 20:58:48,951 INFO
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3123: --- Status: Patch Available (was: Open) Add required apache header and minor test cleanup. Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2011-09-27
[jira] [Updated] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail
[ https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hitesh Shah updated MAPREDUCE-3123: --- Attachment: MR-3123.3.patch Symbolic links with special chars causing container/task.sh to fail --- Key: MAPREDUCE-3123 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Thomas Graves Assignee: Hitesh Shah Priority: Blocker Fix For: 0.23.0 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch the following job throws an exception when you have the special characters in it. hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=* Exception: 2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer: [container-executor, hadoopuser, 1, application_1317077272567_0239, container_1317077272567 0239_01_01, tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01, tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 01/task.sh, tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27 20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2 2011-09-27 20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch : org.apache.hadoop.util.Shell$ExitCodeException: /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: syntax error near unexpected token `-_+=' /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh: line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar: at org.apache.hadoop.util.Shell.runCommand(Shell.java:261) at org.apache.hadoop.util.Shell.run(Shell.java:188) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) 2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: 2011-09-27 20:58:48,951 INFO
