Hi,
I'm using Hadoop 0.20.0 (semidistributed mode, or whatever it's called -- I
can't look up the name, since the documentation on the site seems to be
down), and I'm experiencing a JobTracker crash every time I start Hadoop.
The JobTracker seems to be 'INITIALIZING' for a few seconds, and then goes
down. I then use the bin/start-mapred.sh command to start the JobTracker
again, after which it displays the status 'RUNNING' and works fine.
Now I want to add JUnit tests with MiniMRCluster, and I'm experiencing the
same problem there (only it's not possible to start the JobTracker again).
I'm seeing a lot of these errors in my logs:
2009-07-17 09:32:56,718 WARN org.apache.hadoop.mapred.JobTracker: Error
starting tracker: org.apache.hadoop.ipc.RemoteException:
java.io.IOException: File
/tmp/hadoop-mathias/mapred/system/jobtracker.infocould only be
replicated to 0 nodes, instead of 1
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1256)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
at org.apache.hadoop.ipc.Client.call(Client.java:739)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at $Proxy4.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy4.addBlock(Unknown Source)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:2873)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:2755)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2000(DFSClient.java:2046)
at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2232)
Followed by one:
2009-07-17 09:32:57,719 FATAL org.apache.hadoop.mapred.JobTracker:
java.net.BindException: Problem binding to localhost/127.0.0.1:9001 :
Address already in use
at org.apache.hadoop.ipc.Server.bind(Server.java:190)
at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253)
at org.apache.hadoop.ipc.Server.<init>(Server.java:1026)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488)
at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450)
at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1537)
at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:174)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3528)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:119)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
at org.apache.hadoop.ipc.Server.bind(Server.java:188)
... 7 more
2009-07-17 09:32:57,720 INFO org.apache.hadoop.mapred.JobTracker:
SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down JobTracker at mathias-desktop/127.0.1.1
I'm sure the JobTracker is the only program using that port, so perhaps it's
somehow conflicting with itself?
