Sandy Ryza created HADOOP-8775:
----------------------------------
Summary: MR2 distcp permits non-positive value to -bandwidth
option which causes job never to complete
Key: HADOOP-8775
URL: https://issues.apache.org/jira/browse/HADOOP-8775
Project: Hadoop Common
Issue Type: Bug
Affects Versions: 2.0.0-alpha
Reporter: Sandy Ryza
The likelihood that someone would want to enter a non-positive value for
-bandwidth seems really low. However, the job would never complete if a
non-positive value was specified. It'd just get stuck at map 100%. Luckily, a
positive value would always lead to the job completing.
bash-4.1$ hadoop distcp -bandwidth 0
hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir
hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
hadoop distcp -bandwidth 0 hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir
hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58
12/05/23 15:53:01 INFO tools.DistCp: Input Options:
DistCpOptions{atomicCommit=false, syncFolder=false, deleteMissing=false,
ignoreFailures=false, maxMaps=20, sslConfigurationFile='null',
copyStrategy='uniformsiz\
e', sourceFileListing=null,
sourcePaths=[hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir],
targetPath=hdfs://c1204.hal.cloudera.com:17020/user/hdfs/in-dir58}
12/05/23 15:53:02 WARN conf.Configuration: io.sort.mb is deprecated. Instead,
use mapreduce.task.io.sort.mb
12/05/23 15:53:02 WARN conf.Configuration: io.sort.factor is deprecated.
Instead, use mapreduce.task.io.sort.factor
12/05/23 15:53:02 INFO util.NativeCodeLoader: Loaded the native-hadoop library
12/05/23 15:53:03 INFO mapreduce.JobSubmitter: number of splits:3
12/05/23 15:53:04 WARN conf.Configuration: mapred.jar is deprecated. Instead,
use mapreduce.job.jar
12/05/23 15:53:04 WARN conf.Configuration:
mapred.map.tasks.speculative.execution is deprecated. Instead, use
mapreduce.map.speculative
12/05/23 15:53:04 WARN conf.Configuration: mapred.reduce.tasks is deprecated.
Instead, use mapreduce.job.reduces
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.value.class is
deprecated. Instead, use mapreduce.map.output.value.class
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.map.class is deprecated.
Instead, use mapreduce.job.map.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.job.name is deprecated.
Instead, use mapreduce.job.name
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.inputformat.class is
deprecated. Instead, use mapreduce.job.inputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
12/05/23 15:53:04 WARN conf.Configuration: mapreduce.outputformat.class is
deprecated. Instead, use mapreduce.job.outputformat.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.map.tasks is deprecated.
Instead, use mapreduce.job.maps
12/05/23 15:53:04 WARN conf.Configuration: mapred.mapoutput.key.class is
deprecated. Instead, use mapreduce.map.output.key.class
12/05/23 15:53:04 WARN conf.Configuration: mapred.working.dir is deprecated.
Instead, use mapreduce.job.working.dir
12/05/23 15:53:04 INFO mapred.ResourceMgrDelegate: Submitted application
application_1337808305464_0014 to ResourceManager at
c1204.hal.cloudera.com/172.29.98.195:8040
12/05/23 15:53:04 INFO mapreduce.Job: The url to track the job:
http://auto0:8088/proxy/application_1337808305464_0014/
12/05/23 15:53:04 INFO tools.DistCp: DistCp job-id: job_1337808305464_0014
12/05/23 15:53:04 INFO mapreduce.Job: Running job: job_1337808305464_0014
12/05/23 15:53:09 INFO mapreduce.Job: Job job_1337808305464_0014 running in
uber mode : false
12/05/23 15:53:09 INFO mapreduce.Job: map 0% reduce 0%
12/05/23 15:53:14 INFO mapreduce.Job: map 33% reduce 0%
12/05/23 15:53:19 INFO mapreduce.Job: map 100% reduce 0%
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
