Namenode is not able to recover from disk full condition
--------------------------------------------------------
Key: HADOOP-6774
URL: https://issues.apache.org/jira/browse/HADOOP-6774
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.20.2
Environment: Linux sjc9-flash-grid00.ciq.com 2.6.18-164.el5 #1 SMP Thu
Sep 3 03:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux
Reporter: Ted Yu
We ran an internal flow which resulted in:
Exception in thread "main" java.lang.RuntimeException: initialization of flow
executor failed
After that we freed disk space on the Namenode server, but restarting Namenode
failed.
Here is from Namenode log:
2010-05-19 17:15:15,514 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
Namenode up at: sjc1-qa-certiq1.sjc1.ciq.com/10.201.8.247:9000
2010-05-19 17:15:15,516 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
Initializing JVM Metrics with processName=NameNode, sessionId=null
2010-05-19 17:15:15,518 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.NameNodeMetrics: Initializing
NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext
2010-05-19 17:15:15,579 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=hadoop,hadoop
2010-05-19 17:15:15,579 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup
2010-05-19 17:15:15,579 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true
2010-05-19 17:15:15,588 INFO
org.apache.hadoop.hdfs.server.namenode.metrics.FSNamesystemMetrics:
Initializing FSNamesystemMetrics using context
object:org.apache.hadoop.metrics.spi.NullContext
2010-05-19 17:15:15,590 INFO
org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered
FSNamesystemStatusMBean
2010-05-19 17:15:15,637 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files = 1874
2010-05-19 17:15:16,202 INFO org.apache.hadoop.hdfs.server.common.Storage:
Number of files under construction = 2
2010-05-19 17:15:16,204 INFO org.apache.hadoop.hdfs.server.common.Storage:
Image file of size 259450 loaded in 0 seconds.
2010-05-19 17:15:16,599 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode:
java.lang.NumberFormatException: For input string: ""
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:431)
at java.lang.Long.parseLong(Long.java:468)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.readLong(FSEditLog.java:1273)
at
org.apache.hadoop.hdfs.server.namenode.FSEditLog.loadFSEdits(FSEditLog.java:656)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSEdits(FSImage.java:999)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:812)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:364)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:88)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:312)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:293)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:224)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:306)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1004)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1013)
2010-05-19 17:15:16,599 INFO org.apache.hadoop.hdfs.server.namenode.NameNode:
SHUTDOWN_MSG:
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
