[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-10 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124023#comment-13124023
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Hdfs-trunk #826 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/826/])
MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. 
Contributed by Hitesh Shah.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180772
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

   

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-10 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124052#comment-13124052
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Hdfs-0.23-Build #35 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/35/])
Merge -c 1180772 from trunk to branch-0.23 to fix MAPREDUCE-3123.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180773
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

  

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-10 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124077#comment-13124077
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Mapreduce-0.23-Build #42 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/42/])
Merge -c 1180772 from trunk to branch-0.23 to fix MAPREDUCE-3123.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180773
Files : 
* /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)


[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-10 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13124087#comment-13124087
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Mapreduce-trunk #856 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/856/])
MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. 
Contributed by Hitesh Shah.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180772
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

 

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-09 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13123871#comment-13123871
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Common-trunk-Commit #1049 (See 
[https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1049/])
MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. 
Contributed by Hitesh Shah.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180772
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

   

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-09 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13123875#comment-13123875
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Hdfs-trunk-Commit #1127 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1127/])
MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. 
Contributed by Hitesh Shah.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180772
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

   

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-09 Thread Hudson (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13123888#comment-13123888
 ] 

Hudson commented on MAPREDUCE-3123:
---

Integrated in Hadoop-Mapreduce-trunk-Commit #1069 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1069/])
MAPREDUCE-3123. Fix NM to quote symlink names to escape special characters. 
Contributed by Hitesh Shah.

acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1180772
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

 

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-07 Thread Hitesh Shah (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13123293#comment-13123293
 ] 

Hitesh Shah commented on MAPREDUCE-3123:


Will open a separate jira to track what should be done for escaping user input 
( commands, filenames ) in general before we use/execute them. 

 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

   

 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197)
   
   

 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:619)
 2011-09-27 20:58:48,951 INFO 
 

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-07 Thread Hadoop QA (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13123335#comment-13123335
 ] 

Hadoop QA commented on MAPREDUCE-3123:
--

+1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12498264/MR-3123.3.patch
  against trunk revision .

+1 @author.  The patch does not contain any @author tags.

+1 tests included.  The patch appears to include 2 new or modified tests.

+1 javadoc.  The javadoc tool did not generate any warning messages.

+1 javac.  The applied patch does not increase the total number of javac 
compiler warnings.

+1 findbugs.  The patch does not introduce any new Findbugs (version 1.3.9) 
warnings.

+1 release audit.  The applied patch does not increase the total number of 
release audit warnings.

+1 core tests.  The patch passed unit tests in .

+1 contrib tests.  The patch passed contrib unit tests.

Test results: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/974//testReport/
Console output: 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/974//console

This message is automatically generated.

 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0

 Attachments: MR-3123.1.patch, MR-3123.2.patch, MR-3123.3.patch


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

  

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-10-06 Thread Hitesh Shah (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13122483#comment-13122483
 ] 

Hitesh Shah commented on MAPREDUCE-3123:


Vinod/Thomas/Arun - a query on task.sh - the final command i.e. exec setsid 
/bin/bash -c ... encapsulates the user provided command with . The 
assumption here seems to be they can use environment variables with the command 
for expansion and that the onus of correctly escaping the command i.e escaping 
any special chars is on the users themselves. Would that be correct?

For example, creating the symlink above is pretty simply by just quoting the 
file name but if the symlink is something that will be executed then special 
chars like  or even basic things like spaces can create problems if not 
quoted and escaped properly.  

 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Assignee: Hitesh Shah
Priority: Blocker
 Fix For: 0.23.0


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)
  
   
  
  at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381) 
  
   
 
 at 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

   

 at 
 org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197)
   
   

 at 
 

[jira] [Commented] (MAPREDUCE-3123) Symbolic links with special chars causing container/task.sh to fail

2011-09-29 Thread Thomas Graves (Commented) (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-3123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13117545#comment-13117545
 ] 

Thomas Graves commented on MAPREDUCE-3123:
--

lines from task.sh file:

mkdir -p jobSubmitDir
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317077272567_0239/filecache/-6725992875387940725/job.split
 jobSubmitDir/job.split
mkdir -p jobSubmitDir
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317077272567_0239/filecache/-8476253876386538612/appTokens
 jobSubmitDir/appTokens
mkdir -p jobSubmitDir
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317077272567_0239/filecache/-2497890035284830309/job.splitmetainfo
 jobSubmitDir/job.splitmetainfo
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317077272567_0239/filecache/-8047704159844243328/job.jar
 job.jar
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/appcache/application_1317077272567_0239/filecache/-6010439732820580780/job.xml
 job.xml
ln -sf 
/tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
testlink!@$*()-_+=
exec setsid /bin/bash -c $JAVA_HOME/bin/java 
-Dlog4j.configuration=container-log4j.properties 
-Dyarn.app.mapreduce.container.log.dir=/hadoop/var/log/mapredqa/application_1317077272567_0239/container_1317077272567_0239_01_01
 -Dyarn.app.mapreduce.container.log.filesize=0 -Xmx1024m 
org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1317077272567 239 1 
1/hadoop/var/log/mapredqa/application_1317077272567_0239/container_1317077272567_0239_01_01/stdout
 
2/hadoop/var/log/mapredqa/application_1317077272567_0239/container_1317077272567_0239_01_01/stderr
  


 Symbolic links with special chars causing container/task.sh to fail
 ---

 Key: MAPREDUCE-3123
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3123
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: mrv2
Affects Versions: 0.23.0
Reporter: Thomas Graves
Priority: Blocker
 Fix For: 0.23.0


 the following job throws an exception when you have the special characters in 
 it.
 hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* 
 -Dmapreduce.job.queuename=queue1 -files 
 file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$*()-_+=
  -input Streaming/streaming-980/input.txt  -mapper 'xargs cat' -reducer cat 
 -output Streaming/streaming-980/Output -jobconf 
 mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*
 Exception:
 2011-09-27 20:58:48,903 INFO 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: 
 launchContainer:
 [container-executor, hadoopuser, 1, application_1317077272567_0239,
 container_1317077272567 0239_01_01,
 tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01,
 tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01
  01/task.sh,
 tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_01/container_1317077272567_0239_01_01.tokens]110922-tests.jar:hadoop-mapreduce-p2011-09-27
 20:58:48,944 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code 
 from container is : 2
   
  2011-09-27
 20:58:48,946 WARN 
 org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception 
 from container-launch :  
   
 
 org.apache.hadoop.util.Shell$ExitCodeException:
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: syntax error near unexpected token `-_+='   
 /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_01/task.sh:
 line 26: `ln -sf 
 /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir 
 test
 ink!@$*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:
  at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)   
  
   
  
at org.apache.hadoop.util.Shell.run(Shell.java:188)