distcp -update fails if source directory is empty (i.e. no files to copy) and
target directory does not exists.
---------------------------------------------------------------------------------------------------------------
Key: MAPREDUCE-662
URL: https://issues.apache.org/jira/browse/MAPREDUCE-662
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: distcp
Affects Versions: 0.21.0
Reporter: Suhas Gogate
It should either create empty target directory or not make any changes on the
target (as there is nothing to copy from source), but it should return success.
Tested version of hadoop has HADOOP-5675.
-bash-3.1$ bin/hadoop distcp -Dmapred.job.queue.name=xxx -i -p -update -m 10
hftp://xxx.yyy.com:50070/user/gogate/mirror_test1
hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1
09/06/24 19:46:42 INFO tools.DistCp:
srcPaths=[hftp://xxx.yyy.com:50070/user/gogate/mirror_test1]
09/06/24 19:46:42 INFO tools.DistCp:
destPath=hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1
09/06/24 19:46:44 INFO tools.DistCp:
hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1 does not exist.
09/06/24 19:46:44 INFO tools.DistCp: sourcePathsCount=1
09/06/24 19:46:44 INFO tools.DistCp: filesToCopyCount=0
09/06/24 19:46:44 INFO tools.DistCp: bytesToCopyCount=0.0
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.FileNotFoundException: File does not exist:
hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1
at
org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:457)
at org.apache.hadoop.tools.DistCp.finalize(DistCp.java:706)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:653)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:858)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:885)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
