I've run into an interesting problem with syncing a couple of clusters
using distcp. We've validated that it works to a local installation
from our remote cluster. I suspect our firewalls 'may' be responsible
for the problem we're experiencing. We're using ports 9000, 9001 and
50010.I've verified all three ports are available to the namenodes and
datanodes in both directions. Is there something else we're missing?
Looks like it get's to 80% before it fails. Here's what we're seeing.
# u...@hnn1:~$ hadoop distcp hdfs://hnn1:9000/user/testing
hdfs://hnn2:9000/user
10/12/03 15:58:10 INFO tools.DistCp:
srcPaths=[hdfs://hnn1:9000/user/testing]
10/12/03 15:58:10 INFO tools.DistCp: destPath=hdfs://hnn2:9000/user
10/12/03 15:58:11 INFO tools.DistCp: srcCount=6
10/12/03 15:58:11 INFO mapred.JobClient: Running job: job_201011221457_0019
10/12/03 15:58:12 INFO mapred.JobClient: map 0% reduce 0%
10/12/03 15:58:36 INFO mapred.JobClient: map 19% reduce 0%
10/12/03 15:58:45 INFO mapred.JobClient: map 39% reduce 0%
10/12/03 15:59:03 INFO mapred.JobClient: map 60% reduce 0%
10/12/03 15:59:12 INFO mapred.JobClient: map 80% reduce 0%
10/12/03 15:59:32 INFO mapred.JobClient: Task Id :
attempt_201011221457_0019_m_000000_0, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 5
at
org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:572)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
10/12/03 15:59:33 INFO mapred.JobClient: map 0% reduce 0%
10/12/03 15:59:55 INFO mapred.JobClient: map 19% reduce 0%
10/12/03 16:00:04 INFO mapred.JobClient: map 39% reduce 0%
10/12/03 16:00:22 INFO mapred.JobClient: map 60% reduce 0%
10/12/03 16:00:31 INFO mapred.JobClient: map 80% reduce 0%
10/12/03 16:00:51 INFO mapred.JobClient: Task Id :
attempt_201011221457_0019_m_000000_1, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 5
at
org.apache.hadoop.tools.DistCp$CopyFilesMapper.close(DistCp.java:572)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Thanks!
