This got fixed when the directory referring with hadoop.tmp.dir in core-site.xml is cleaned up and then the namenode is formatted.
On Wed, Aug 8, 2012 at 10:25 AM, Deependra Ariyadewa <d...@wso2.com> wrote: > Hi Chamara, > > Pl try to start the name node without all the other services and attach > the name node log. It will help to isolate the issue in Name Node. > > You can start the name node by executing "bin/hadoop namenode" > > > Thanks, > Deependra. > > On Wed, Aug 8, 2012 at 9:56 AM, Chamara Ariyarathne <chama...@wso2.com>wrote: > >> I get the following [1] when starting hadoop namenode in the distributed >> hadoop cluster and it prevents from starting all the hadoop nodes. >> >> /etc/hosts is: >> 10.100.3.221 bamhadoop01 >> 10.100.3.228 bamhadoop02 >> 10.100.3.224 bamhadoop03 >> 10.100.3.244 appserver.cloud-test.wso2.com >> >> # The following lines are desirable for IPv6 capable hosts >> ::1 ip6-localhost ip6-loopback >> fe00::0 ip6-localnet >> ff00::0 ip6-mcastprefix >> ff02::1 ip6-allnodes >> ff02::2 ip6-allrouters >> >> $ hostname >> bamhadoop01 >> >> Property in core-site.xml >> <property> >> <name>fs.default.name</name> >> <value>hdfs://bamhadoop01:9000</value> >> </property> >> <property> >> >> Can ssh to any other hadoop datanode machines without password. >> >> [1] >> 2012-08-08 09:45:05,289 INFO >> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from >> hadoop-metrics2.properties >> 2012-08-08 09:45:05,297 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> MetricsSystem,sub=Stats registered. >> 2012-08-08 09:45:05,297 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot >> period at 10 second(s). >> 2012-08-08 09:45:05,297 INFO >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: JobTracker metrics >> system started >> 2012-08-08 09:45:05,478 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi >> registered. >> 2012-08-08 09:45:05,481 WARN >> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already >> exists! >> 2012-08-08 09:45:05,482 INFO >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: >> Updating the current master key for generating delegation tokens >> 2012-08-08 09:45:05,483 INFO >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: >> Starting expired delegation token remover thread, >> tokenRemoverScanInterval=60 min(s) >> 2012-08-08 09:45:05,483 INFO >> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager: >> Updating the current master key for generating delegation tokens >> 2012-08-08 09:45:05,483 INFO org.apache.hadoop.mapred.JobTracker: >> Scheduler configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT, >> limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1) >> 2012-08-08 09:45:05,484 INFO org.apache.hadoop.util.HostsFileReader: >> Refreshing hosts (include/exclude) list >> 2012-08-08 09:45:05,517 INFO org.apache.hadoop.mapred.JobTracker: >> Starting jobtracker with owner as bamtesttmp >> 2012-08-08 09:45:05,531 INFO org.apache.hadoop.ipc.Server: Starting >> SocketReader >> 2012-08-08 09:45:05,533 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> RpcDetailedActivityForPort9001 registered. >> 2012-08-08 09:45:05,534 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> RpcActivityForPort9001 registered. >> 2012-08-08 09:45:10,598 INFO org.mortbay.log: Logging to >> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> org.mortbay.log.Slf4jLog >> 2012-08-08 09:45:10,634 INFO org.apache.hadoop.http.HttpServer: Added >> global filtersafety >> (class=org.apache.hadoop.http.HttpServer$QuotingInputFilter) >> 2012-08-08 09:45:10,655 INFO org.apache.hadoop.http.HttpServer: Port >> returned by webServer.getConnectors()[0].getLocalPort() before open() is >> -1. Opening the listener on 50030 >> 2012-08-08 09:45:10,656 INFO org.apache.hadoop.http.HttpServer: >> listener.getLocalPort() returned 50030 >> webServer.getConnectors()[0].getLocalPort() returned 50030 >> 2012-08-08 09:45:10,656 INFO org.apache.hadoop.http.HttpServer: Jetty >> bound to port 50030 >> 2012-08-08 09:45:10,656 INFO org.mortbay.log: jetty-6.1.26 >> 2012-08-08 09:45:10,676 WARN org.mortbay.log: Can't reuse >> /tmp/Jetty_0_0_0_0_50030_job____yn7qmk, using >> /tmp/Jetty_0_0_0_0_50030_job____yn7qmk_2518359383055609936 >> 2012-08-08 09:45:10,811 INFO org.mortbay.log: Started >> SelectChannelConnector@0.0.0.0:50030 >> 2012-08-08 09:45:10,815 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm >> registered. >> 2012-08-08 09:45:10,815 INFO >> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source >> JobTrackerMetrics registered. >> 2012-08-08 09:45:10,815 INFO org.apache.hadoop.mapred.JobTracker: >> JobTracker up at: 9001 >> 2012-08-08 09:45:10,815 INFO org.apache.hadoop.mapred.JobTracker: >> JobTracker webserver: 50030 >> 2012-08-08 09:45:11,867 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 0 >> time(s). >> 2012-08-08 09:45:12,868 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 1 >> time(s). >> 2012-08-08 09:45:13,868 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 2 >> time(s). >> 2012-08-08 09:45:14,869 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 3 >> time(s). >> 2012-08-08 09:45:15,869 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 4 >> time(s). >> 2012-08-08 09:45:16,870 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 5 >> time(s). >> 2012-08-08 09:45:17,870 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 6 >> time(s). >> 2012-08-08 09:45:18,871 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 7 >> time(s). >> 2012-08-08 09:45:19,871 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 8 >> time(s). >> 2012-08-08 09:45:20,872 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 9 >> time(s). >> 2012-08-08 09:45:20,874 INFO org.apache.hadoop.mapred.JobTracker: problem >> cleaning system directory: null >> java.net.ConnectException: Call to bamhadoop01/10.100.3.221:9000 failed >> on connection exception: java.net.ConnectException: Connection refused >> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1057) >> at org.apache.hadoop.ipc.Client.call(Client.java:1033) >> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224) >> at $Proxy5.getProtocolVersion(Unknown Source) >> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:364) >> at >> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106) >> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:208) >> at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:175) >> at >> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89) >> at >> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1310) >> at org.apache.hadoop.fs.FileSystem.access$100(FileSystem.java:65) >> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1328) >> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:226) >> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:109) >> at org.apache.hadoop.mapred.JobTracker$3.run(JobTracker.java:2349) >> at org.apache.hadoop.mapred.JobTracker$3.run(JobTracker.java:2347) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059) >> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2347) >> at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:2171) >> at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:300) >> at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:291) >> at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4956) >> Caused by: java.net.ConnectException: Connection refused >> at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >> at >> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) >> at >> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) >> at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:406) >> at >> org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:414) >> at >> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:527) >> at org.apache.hadoop.ipc.Client$Connection.access$1800(Client.java:187) >> at org.apache.hadoop.ipc.Client.getConnection(Client.java:1164) >> at org.apache.hadoop.ipc.Client.call(Client.java:1010) >> ... 22 more >> 2012-08-08 09:45:31,877 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 0 >> time(s). >> 2012-08-08 09:45:32,878 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 1 >> time(s). >> 2012-08-08 09:45:33,878 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 2 >> time(s). >> 2012-08-08 09:45:34,879 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 3 >> time(s). >> 2012-08-08 09:45:35,879 INFO org.apache.hadoop.ipc.Client: Retrying >> connect to server: bamhadoop01/10.100.3.221:9000. Already tried 4 >> time(s). >> >> >> >> -- >> *Chamara Ariyarathne* >> Senior Software Engineer - QA; >> WSO2 Inc; http://www.wso2.com/. >> Mobile; *+94772786766* >> >> >> _______________________________________________ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > Deependra Ariyadewa > WSO2, Inc. http://wso2.com/ http://wso2.org > > email d...@wso2.com; cell +94 71 403 5996 ; > Blog http://risenfall.wordpress.com/ > PGP info: KeyID: 'DC627E6F' > > -- *Chamara Ariyarathne* Senior Software Engineer - QA; WSO2 Inc; http://www.wso2.com/. Mobile; *+94772786766*
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
