The JobConf instance passed to JobClient is unnecessarily modified
------------------------------------------------------------------
Key: HADOOP-701
URL: http://issues.apache.org/jira/browse/HADOOP-701
Project: Hadoop
Issue Type: Bug
Components: mapred
Affects Versions: 0.8.0
Reporter: Nigel Daley
The JobConf instance passed to the JobClient.submitJob method is modified by
that method. This side effect is surprising. IMHO, the passed in JobConf
should either be defensively copied or the side effect should be documented.
Trying to reuse a JobConf instance in multiple calls to JobClient.submitJob
causes the following exception:
java.io.IOException: /dfstmp/mapred/system/submit_r7mxti/job.jar: No such file
or directory
at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:80)
at
org.apache.hadoop.dfs.DistributedFileSystem.copyFromLocalFile(DistributedFileSystem.java:188)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:291)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:368)
at org.apache.hadoop.examples.NNBench.main(NNBench.java:274)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:41)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.apache.hadoop.util.RunJar.main(RunJar.java:149)
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
