The primary namenode on my cluster seems to have stopped working. The secondary name node starts, but the primary fails with the error message below.
I've scoured the cluster, particularly this node for changes, but I haven't found any that I believe would cause this problem. If anyone has an idea what I might look for, I'd appreciate any help. Also, is there any way to increase the verbosity of the logging? -Colin ------- ************************************************************/ 2008-05-09 11:31:46,484 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = dev04/10.0.2.12 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.16.1 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.16 -r 635123; compiled by 'hadoopqa' on Sun Mar 9 05:44:19 UTC 2008 ************************************************************/ 2008-05-09 11:31:46,656 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=54310 2008-05-09 11:31:46,665 INFO org.apache.hadoop.dfs.NameNode: Namenode up at: dev04/10.0.2.12:54310 2008-05-09 11:31:46,671 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null 2008-05-09 11:31:46,676 INFO org.apache.hadoop.dfs.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext 2008-05-09 11:31:46,761 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=hadoop,hadoop 2008-05-09 11:31:46,761 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup 2008-05-09 11:31:46,761 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true 2008-05-09 11:31:47,132 INFO org.apache.hadoop.ipc.Server: Stopping server on 54310 2008-05-09 11:31:47,135 ERROR org.apache.hadoop.dfs.NameNode: java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:178) at org.apache.hadoop.io.UTF8.readFields(UTF8.java:106) at org.apache.hadoop.io.ArrayWritable.readFields(ArrayWritable.java:90) at org.apache.hadoop.dfs.FSEditLog.loadFSEdits(FSEditLog.java:433) at org.apache.hadoop.dfs.FSImage.loadFSEdits(FSImage.java:756) at org.apache.hadoop.dfs.FSImage.loadFSImage(FSImage.java:639) at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:222) at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:79) at org.apache.hadoop.dfs.FSNamesystem.initialize(FSNamesystem.java:254) at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:235) at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:131) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:176) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:162) at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:846) at org.apache.hadoop.dfs.NameNode.main(NameNode.java:855) 2008-05-09 11:31:47,135 INFO org.apache.hadoop.dfs.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at dev04/10.0.2.12 ************************************************************/
