Hello again,
In fact I can get the cluster up and running with two nodes in different
LANs. The problem appears when executing a job.
As you can see in the piece of log bellow, the datanode tries to comunicate
with the namenode using the IP 10.1.1.5. The issue is that the datanode
should be using a valid IP, and not 10.1.1.5.
Is there a way of manually configuring the datanode with the namenode's IP,
so I can change from 10.1.1.5 to, say 189.11.131.172?
Thanks,
Lucas
2008-08-08 02:34:23,335 INFO org.apache.hadoop.mapred.TaskTracker:
TaskTracker up at: localhost/127.0.0.1:60394
2008-08-08 02:34:23,335 INFO org.apache.hadoop.mapred.TaskTracker: Starting
tracker tracker_localhost:localhost/127.0.0.1:60394
2008-08-08 02:34:23,589 INFO org.apache.hadoop.mapred.TaskTracker: Starting
thread: Map-events fetcher for all reduce tasks on
tracker_localhost:localhost/127.0.0.1:60394
2008-08-08 03:06:43,239 INFO org.apache.hadoop.mapred.TaskTracker:
LaunchTaskAction: task_200808080234_0001_m_000000_0
2008-08-08 03:07:43,989 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 1 time(s).
2008-08-08 03:08:44,999 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 2 time(s).
2008-08-08 03:09:45,999 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 3 time(s).
2008-08-08 03:10:47,009 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 4 time(s).
2008-08-08 03:11:48,009 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 5 time(s).
2008-08-08 03:12:49,026 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 6 time(s).
2008-08-08 03:13:50,036 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 7 time(s).
2008-08-08 03:14:51,046 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 8 time(s).
2008-08-08 03:15:52,056 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 9 time(s).
2008-08-08 03:16:53,066 INFO org.apache.hadoop.ipc.Client: Retrying connect
to server: /10.1.1.5:9000. Already tried 10 time(s).
2008-08-08 03:17:54,077 WARN org.apache.hadoop.mapred.TaskTracker: Error
initializing task_200808080234_0001_m_000000_0:
java.net.SocketTimeoutException
at sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:109)
at
org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:174)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:623)
at org.apache.hadoop.ipc.Client.call(Client.java:546)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:212)
at org.apache.hadoop.dfs.$Proxy5.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:313)
at org.apache.hadoop.dfs.DFSClient.createRPCNamenode(DFSClient.java:102)
at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:178)
at
org.apache.hadoop.dfs.DistributedFileSystem.initialize(DistributedFileSystem.java:68)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1280)
at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:56)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1291)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
at org.apache.hadoop.fs.FileSystem.getNamed(FileSystem.java:152)
at
org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:670)
at
org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
at
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
at org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
On Fri, Aug 8, 2008 at 12:16 AM, Lucas Nazário dos Santos <
[EMAIL PROTECTED]> wrote:
> Hello,
>
> Can someone point me out what are the extra tasks that need to be performed
> in order to set up a cluster where nodes are spread over the Internet, in
> different LANs?
>
> Do I need to free any datanode/namenode ports? How do I get the datanodes
> to know the valid namenode IP, and not something like 10.1.1.1?
>
> Any help is appreciate.
>
> Lucas
>
