[jira] [Updated] (MAPREDUCE-4793) Problem with adding resources when using both -files and -file to hadoop streaming
[ https://issues.apache.org/jira/browse/MAPREDUCE-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4793: -- Attachment: MAPREDUCE-4793.patch Straigthforward fix that I manually tested. Problem with adding resources when using both -files and -file to hadoop streaming -- Key: MAPREDUCE-4793 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4793 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.3 Reporter: Thomas Graves Attachments: MAPREDUCE-4793.patch It seems when both -files and -file are present, it will trigger this IAE, and the error message is just misleading. hadoop jar $HADOOP_PREFIX/share/hadoop/tools/lib/hadoop-streaming.jar -files hdfs://host:port/user/foo/access.log#test -input 'input' -output 'output' -mapper egrep '.*' -file tmp.file Below is the error message 12/11/02 07:37:54 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/haiyang/.staging/job_1351804437209_0575 Exception in thread main java.lang.IllegalArgumentException: Resource name must be relative at org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:383) at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:324) at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419) at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4793) Problem with adding resources when using both -files and -file to hadoop streaming
[ https://issues.apache.org/jira/browse/MAPREDUCE-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4793: -- Assignee: Jason Lowe Target Version/s: 2.0.3-alpha, 0.23.6 Status: Patch Available (was: Open) Problem with adding resources when using both -files and -file to hadoop streaming -- Key: MAPREDUCE-4793 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4793 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.3 Reporter: Thomas Graves Assignee: Jason Lowe Attachments: MAPREDUCE-4793.patch It seems when both -files and -file are present, it will trigger this IAE, and the error message is just misleading. hadoop jar $HADOOP_PREFIX/share/hadoop/tools/lib/hadoop-streaming.jar -files hdfs://host:port/user/foo/access.log#test -input 'input' -output 'output' -mapper egrep '.*' -file tmp.file Below is the error message 12/11/02 07:37:54 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/haiyang/.staging/job_1351804437209_0575 Exception in thread main java.lang.IllegalArgumentException: Resource name must be relative at org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:383) at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:324) at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419) at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4793) Problem with adding resources when using both -files and -file to hadoop streaming
[ https://issues.apache.org/jira/browse/MAPREDUCE-4793?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-4793: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) I committed this to trunk, branch-2, and branch-0.23. Problem with adding resources when using both -files and -file to hadoop streaming -- Key: MAPREDUCE-4793 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4793 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.23.3 Reporter: Thomas Graves Assignee: Jason Lowe Attachments: MAPREDUCE-4793.patch It seems when both -files and -file are present, it will trigger this IAE, and the error message is just misleading. hadoop jar $HADOOP_PREFIX/share/hadoop/tools/lib/hadoop-streaming.jar -files hdfs://host:port/user/foo/access.log#test -input 'input' -output 'output' -mapper egrep '.*' -file tmp.file Below is the error message 12/11/02 07:37:54 INFO mapreduce.JobSubmitter: Cleaning up the staging area /user/haiyang/.staging/job_1351804437209_0575 Exception in thread main java.lang.IllegalArgumentException: Resource name must be relative at org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:383) at org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:324) at org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419) at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
