Additional, If these are two new clusters, then on each namenode, using "hdfs namenode -format -clusterID yourID"
But if you want to upgrade these two clusters from NonHA to HA, then using "bin/start-dfs.sh -upgrade -clusterID yourID" On Thu, Jul 4, 2013 at 3:14 PM, Azuryy Yu <azury...@gmail.com> wrote: > > This is because you don't use the same clusterID. all data nodes and > namenodes should use the same clusterID. > > > On Thu, Jul 4, 2013 at 3:12 PM, Bing Jiang <jiangbinglo...@gmail.com>wrote: > >> Hi, all >> >> We try to use hadoop-2.0.5-alpha, using two namespaces, one is for hbase >> cluster, and the other one is for common use.At the same time, we use >> Quorum Journal policy as HA. >> >> GS-CIX-SEV0001, GS-CIX-SEV0002, namenodes in hbasecluster namespace >> >> GS-CIX-SEV0003, GS-CIX-SEV0004, namenodes in commoncluster namespace. >> >> GS-CIX-SEV0001~GS-CIX-SEV0008 , 8 machines used as Datanode >> >> After launching the hdfs cluster all, there is something which makes me >> confused, that each namespace has half of the datanodes. >> >> NameNode 'GS-CIX-SEV0004:9100' >> >> Started: Thu Jul 04 10:28:00 CST 2013 >> Version: 2.0.5-alpha, 1488459 >> Compiled: 2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha >> Cluster ID: CID-15c48d78-2137-4c6e-aacf-0edbf2bb3db7 >> Block Pool ID: BP-1792015895-10.100.2.3-1372904504940 >> >> >> Browse the filesystem >> NameNode Logs >> Go back to DFS home >> >> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ >> Live Datanodes : 4 >> >> >> Last Admin Configured Used Non DFS Remaining >> Used Used Remaining Block Block Pool Failed >> Node Contact State Capacity (GB) Used (GB) >> (%) (%) (%) Blocks Pool Used (%)> Volumes >> (GB) >> (GB) Used (GB) Blocks >> GS-CIX-SEV0001 1 In Service 888.07 0.00 116.04 772.03 >> 0.00 ┌────────────┐ 86.93 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0002 1 In Service 888.07 0.00 135.50 752.57 >> 0.00 ┌────────────┐ 84.74 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0005 1 In Service 888.07 0.00 97.61 790.46 >> 0.00 ┌────────────┐ 89.01 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0006 1 In Service 888.07 0.00 122.30 765.77 >> 0.00 ┌────────────┐ 86.23 0 0.00 0.00 0 >> >> └────────────┘ >> >> >> Another Namespace's NameNode: >> >> NameNode 'GS-CIX-SEV0001:9100' >> >> Started: Thu Jul 04 10:19:03 CST 2013 >> Version: 2.0.5-alpha, 1488459 >> Compiled: 2013-06-01T04:05Z by jenkins from branch-2.0.5-alpha >> Cluster ID: CID-1a53483d-000e-4726-aef1-f500bedb1df6 >> Block Pool ID: BP-1142418822-10.100.2.1-1372904314309 >> >> >> Browse the filesystem >> NameNode Logs >> Go back to DFS home >> >> ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ >> Live Datanodes : 4 >> >> >> Last Admin Configured Used Non DFS Remaining >> Used Used Remaining Block Block Pool Failed >> Node Contact State Capacity (GB) Used (GB) >> (%) (%) (%) Blocks Pool Used (%)> Volumes >> (GB) >> (GB) Used (GB) Blocks >> GS-CIX-SEV0003 0 In Service 888.07 0.00 150.54 737.53 >> 0.00 ┌────────────┐ 83.05 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0004 0 In Service 888.07 0.00 177.22 710.85 >> 0.00 ┌────────────┐ 80.04 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0007 0 In Service 888.07 0.00 62.91 825.16 >> 0.00 ┌────────────┐ 92.92 0 0.00 0.00 0 >> >> └────────────┘ >> GS-CIX-SEV0008 0 In Service 888.07 0.00 125.25 762.82 >> 0.00 ┌────────────┐ 85.90 0 0.00 0.00 0 >> >> >> >> And check the DN(GS-CIX-SEV0001)'s log, it prints like this: >> 2013-07-04 10:34:51,699 FATAL >> org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for >> block pool Block pool BP-1142418822-10.100.2.1-1372904314309 (storage id >> DS-1677272131-10.100.2.1-50010-1372905291690) service to GS-CIX-SEV0001/ >> 10.100.2.1:9100 >> java.io.IOException: Inconsistent storage IDs. Name-node returned >> DS811369792. Expecting DS-1677272131-10.100.2.1-50010-1372905291690 >> at >> org.apache.hadoop.hdfs.server.datanode.DataNode.bpRegistrationSucceeded(DataNode.java:731) >> at >> org.apache.hadoop.hdfs.server.datanode.BPOfferService.registrationSucceeded(BPOfferService.java:308) >> at >> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.register(BPServiceActor.java:632) >> at >> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:225) >> at >> org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:664) >> at java.lang.Thread.run(Thread.java:662) >> >> It is proved that one datanode has been required to attached to only one >> namespace? >> >> Any views about it will be thankful. >> >> Regards~ >> >> >> -- >> Bing Jiang >> Tel：(86)134-2619-1361 >> weibo: http://weibo.com/jiangbinglover >> BLOG: http://blog.sina.com.cn/jiangbinglover >> National Research Center for Intelligent Computing Systems >> Institute of Computing technology >> Graduate University of Chinese Academy of Science >> > >
