Hi, I followed the instructions from http://hayesdavis.net/2008/06/14/running-hadoop-on-windows/ to install Hadoop 0.17.1 on my Windows XP computer, whose computer name is AMBER, and the current user name is User. I installed CygWin on G:\. I have verified ssh and bin/hadoop version work fine. But when trying to start the dfs service I found the following problems:
1. Hadoop can't create the logs directory automatically if it does not exist in the install directory. 2. The datanode service can automatically create the G:\tmp\hadoop-SYSTEM\dfs\data directory, but the namenode service cant' automatically create G:\tmp\hadoop-User directory and it's sub directories, even I manually created the G:\tmp\hadoop-User\dfs\name\image directory the name service can't start neither, I found the following exceptions in the nameservice's log file: 2008-07-18 22:11:46,578 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = amber/116.76.140.27 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.17.1 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.17 -r 669344; compiled by 'hadoopqa' on Thu Jun 19 01:18:25 UTC 2008 ************************************************************/ 2008-07-18 22:11:47,234 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=47110 2008-07-18 22:11:47,250 INFO org.apache.hadoop.dfs.NameNode: Namenode up at: localhost/127.0.0.1:47110 2008-07-18 22:11:47,265 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null 2008-07-18 22:11:47,281 INFO org.apache.hadoop.dfs.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext 2008-07-18 22:11:48,296 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=User,None,root,Administrators,Users,ORA_DBA 2008-07-18 22:11:48,296 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup 2008-07-18 22:11:48,296 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true 2008-07-18 22:11:48,359 INFO org.apache.hadoop.dfs.Storage: Storage directory G:\tmp\hadoop-User\dfs\name does not exist. 2008-07-18 22:11:48,359 INFO org.apache.hadoop.ipc.Server: Stopping server on 47110 2008-07-18 22:11:48,359 ERROR org.apache.hadoop.dfs.NameNode: org.apache.hadoop.dfs.InconsistentFSStateException: Directory G:\tmp\hadoop-User\dfs\name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:154) at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:80) at org.apache.hadoop.dfs.FSNamesystem.initialize(FSNamesystem.java:274) at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:255) at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:133) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:178) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:164) at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:848) at org.apache.hadoop.dfs.NameNode.main(NameNode.java:857) 2008-07-18 22:11:48,359 INFO org.apache.hadoop.dfs.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at amber/116.76.140.27 ************************************************************/ 2008-07-18 22:26:35,734 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = amber/116.76.140.27 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.17.1 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.17 -r 669344; compiled by 'hadoopqa' on Thu Jun 19 01:18:25 UTC 2008 ************************************************************/ 2008-07-18 22:26:36,046 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=47110 2008-07-18 22:26:36,062 INFO org.apache.hadoop.dfs.NameNode: Namenode up at: localhost/127.0.0.1:47110 2008-07-18 22:26:36,062 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null 2008-07-18 22:26:36,093 INFO org.apache.hadoop.dfs.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext 2008-07-18 22:26:37,421 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=User,None,root,Administrators,Users,ORA_DBA 2008-07-18 22:26:37,421 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup 2008-07-18 22:26:37,421 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true 2008-07-18 22:26:37,515 INFO org.apache.hadoop.dfs.Storage: Storage directory G:\tmp\hadoop-User\dfs\name does not exist. 2008-07-18 22:26:37,515 INFO org.apache.hadoop.ipc.Server: Stopping server on 47110 2008-07-18 22:26:37,515 ERROR org.apache.hadoop.dfs.NameNode: org.apache.hadoop.dfs.InconsistentFSStateException: Directory G:\tmp\hadoop-User\dfs\name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:154) at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:80) at org.apache.hadoop.dfs.FSNamesystem.initialize(FSNamesystem.java:274) at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:255) at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:133) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:178) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:164) at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:848) at org.apache.hadoop.dfs.NameNode.main(NameNode.java:857) 2008-07-18 22:26:37,515 INFO org.apache.hadoop.dfs.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at amber/116.76.140.27 ************************************************************/ 2008-07-18 22:35:17,093 INFO org.apache.hadoop.dfs.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = amber/116.76.140.27 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.17.1 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/core/branches/branch-0.17 -r 669344; compiled by 'hadoopqa' on Thu Jun 19 01:18:25 UTC 2008 ************************************************************/ 2008-07-18 22:35:17,328 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics with hostName=NameNode, port=47110 2008-07-18 22:35:17,343 INFO org.apache.hadoop.dfs.NameNode: Namenode up at: localhost/127.0.0.1:47110 2008-07-18 22:35:17,359 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=NameNode, sessionId=null 2008-07-18 22:35:17,359 INFO org.apache.hadoop.dfs.NameNodeMetrics: Initializing NameNodeMeterics using context object:org.apache.hadoop.metrics.spi.NullContext 2008-07-18 22:35:18,000 INFO org.apache.hadoop.fs.FSNamesystem: fsOwner=User,None,root,Administrators,Users,ORA_DBA 2008-07-18 22:35:18,000 INFO org.apache.hadoop.fs.FSNamesystem: supergroup=supergroup 2008-07-18 22:35:18,000 INFO org.apache.hadoop.fs.FSNamesystem: isPermissionEnabled=true 2008-07-18 22:35:18,171 INFO org.apache.hadoop.ipc.Server: Stopping server on 47110 2008-07-18 22:35:18,171 ERROR org.apache.hadoop.dfs.NameNode: java.io.EOFException at java.io.RandomAccessFile.readInt(RandomAccessFile.java:713) at org.apache.hadoop.dfs.FSImage.isConversionNeeded(FSImage.java:488) at org.apache.hadoop.dfs.Storage$StorageDirectory.analyzeStorage(Storage.java:283) at org.apache.hadoop.dfs.FSImage.recoverTransitionRead(FSImage.java:149) at org.apache.hadoop.dfs.FSDirectory.loadFSImage(FSDirectory.java:80) at org.apache.hadoop.dfs.FSNamesystem.initialize(FSNamesystem.java:274) at org.apache.hadoop.dfs.FSNamesystem.<init>(FSNamesystem.java:255) at org.apache.hadoop.dfs.NameNode.initialize(NameNode.java:133) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:178) at org.apache.hadoop.dfs.NameNode.<init>(NameNode.java:164) at org.apache.hadoop.dfs.NameNode.createNameNode(NameNode.java:848) at org.apache.hadoop.dfs.NameNode.main(NameNode.java:857) 2008-07-18 22:35:18,171 INFO org.apache.hadoop.dfs.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at amber/116.76.140.27 ************************************************************/
