Thanks. The fs.default.name is "file:///" and dfs.http.address is
"0.0.0.0:50070". I tried:
hadoop dfs -ls /path/file to make sure file exists on cluster1
hadoop distcp file:///cluster1_master_node_ip:50070/path/file
file:///cluster2_master_node_ip:50070/path/file
It gives this error message:
08/10/22 15:43:47 INFO util.CopyFiles:
srcPaths=[file:/cluster1_master_node_ip:50070/path/file]
08/10/22 15:43:47 INFO util.CopyFiles:
destPath=file:/cluster2_master_node_ip:50070/path/file
With failures, global counters are inaccurate; consider running with -i
Copy failed: org.apache.hadoop.mapred.InvalidInputException: Input source
file:/cluster1_master_node_ip:50070/path/file does not exist.
at org.apache.hadoop.util.CopyFiles.checkSrcPath(CopyFiles.java:578)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:594)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
If I use hdfs:// instead of file:///, I get:
Copy failed: java.net.SocketTimeoutException: timed out waiting for rpc
response
at org.apache.hadoop.ipc.Client.call(Client.java:559)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:313)
at
org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.java:102)
at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:178)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem.java:68)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1280)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:56)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1291)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
at org.apache.hadoop.util.CopyFiles.checkSrcPath(CopyFiles.java:572)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:594)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:743)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:763)
s29752-hadoopuser wrote:
>
> Hi,
>
> There is no such thing called distcp port. distcp uses (generic) file
> system API and so it does not care about the file system implementation
> details like port number.
>
> It is common to use distcp with HDFS or HFTP. The urls will look like
> hdfs://namenode:port/path and hftp://namenode:port/path for HDFS and HFTP,
> respectively. The HDFS and HFTP ports are specified by fs.default.name
> and dfs.http.address, respectively.
>
> Nicholas Sze
>
>
>
>
> ----- Original Message ----
>> From: bzheng <[EMAIL PROTECTED]>
>> To: core-user@hadoop.apache.org
>> Sent: Wednesday, October 22, 2008 11:57:43 AM
>> Subject: distcp port for 0.17.2
>>
>>
>> What's the port number for distcp in 0.17.2? I can't find any
>> documentation
>> on distcp for version 0.17.2. For version 0.18, the documentation says
>> it's
>> 8020.
>>
>> I'm using a standard install and the only open ports associated with
>> hadoop
>> are 50030, 50070, and 50090. None of them work with distcp. So, how do
>> you
>> use distcp in 0.17.2? are there any extra setup/configuration needed?
>>
>> Thanks in advance for your help.
>> --
>> View this message in context:
>> http://www.nabble.com/distcp-port-for-0.17.2-tp20117463p20117463.html
>> Sent from the Hadoop core-user mailing list archive at Nabble.com.
>
>
>
--
View this message in context:
http://www.nabble.com/distcp-port-for-0.17.2-tp20117463p20121246.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.
