[
https://issues.apache.org/jira/browse/MAPREDUCE-5960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14160950#comment-14160950
]
Jason Lowe commented on MAPREDUCE-5960:
---------------------------------------
Part of the issue is that Hadoop only assumes that the cluster nodes can get to
the default filesystem, hence why it's checking if the file is on the same
filesystem as the default filesystem. If not, it copies to that filesystem to
ensure the nodes can access it. I'm not sure it's always safe to assume that
if the client can access a non-"file:" filesystem that the nodes will be able
to as well.
> JobSubmitter's check whether job.jar is local is incorrect with no authority
> in job jar path.
> ---------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5960
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5960
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
> Affects Versions: 2.4.0, 2.4.1
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Attachments: MAPREDUCE-5960.v01.patch
>
>
> {code}
> $ mapred job -submit myjob.xml
> 2014-07-06 15:06:34.731 java[12120:1903] Unable to load realm info from
> SCDynamicStore
> 14/07/06 15:06:34 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> 14/07/06 15:06:34 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not
> <property>
> 14/07/06 15:06:37 WARN conf.Configuration: bad conf file: element not
> <property>
> 14/07/06 15:06:37 INFO client.RMProxy: Connecting to ResourceManager at
> /0.0.0.0:8032
> 14/07/06 15:11:11 INFO mapreduce.JobSubmitter: Cleaning up the staging area
> /tmp/hadoop-yarn/staging/gshegalov/.staging/job_1404679996131_0004
> Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS:
> viewfs:/user/gshegalov/hadoop-mapreduce-examples-3.0.0-SNAPSHOT.jar,
> expected: file:///
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:643)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:80)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:525)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:739)
> at
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:520)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
> at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
> at
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1902)
> at
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1870)
> at
> org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1835)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyJar(JobSubmitter.java:286)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:254)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:301)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:389)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
> at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:260)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1239)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
