hi, Jean.
Thanks a lot.
I am really an idiot of Hbase.
I removed the /hbase root directory from HDFS once, hoping it
will rebuild the whole META-regions thing. Then I found the exception is
still there every time I use the shell command.
Before all that I am gonna ask, I have one question :"Is it OK if
we run hbase shell command on any slave/region server?
I have checked the log, seems the master will request the wrong
regionserver for a region it's not servicing:
2010-01-12 20:25:11,996 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 3 on 60020, call getRegionInfo([...@dc9766) from 192.168.1.98:55351:
error: org.apache.hadoop.hbase.NotServingRegionException: -ROOT-,,0
org.apache.hadoop.hbase.NotServingRegionException: -ROOT-,,0
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:2309)
I am still analyzing the master log, for the most recent start,
there seems no exception records in the log.
On Wed, Jan 13, 2010 at 9:39 AM, Jean-Daniel Cryans <[email protected]>wrote:
> It seems it found the ROOT region but META wasn't assigned. Either you
> didn't wait enough after starting hbase or you should look at the
> master's log for the reason why that region wasn't assigned.
>
> J-D
>
> On Tue, Jan 12, 2010 at 5:36 PM, steven zhuang <[email protected]>
> wrote:
> > That's done, thanks, Jean.
> >
> > But now there is another problem. Now I can start the cluster
> > without any exception(good!), but at any node, when I run list/create, I
> > always get this exception, although afterwards I checked the table is
> > created.
> >
> > 10/01/12 20:25:16 DEBUG client.HConnectionManager$TableServers: Found
> ROOT
> > at 192.168.1.104:60020
> > 10/01/12 20:25:16 DEBUG client.HConnectionManager$TableServers:
> > locateRegionInMeta attempt 0 of 5 failed; retrying after sleep of 2000
> > org.apache.hadoop.hbase.client.NoServerForRegionException: No server
> address
> > listed in -ROOT- for region .META.,,1
> > at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:668)
> > at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
> > at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:563)
> > at
> >
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
> >
> >
> >
> > On Wed, Jan 13, 2010 at 8:57 AM, Jean-Daniel Cryans <[email protected]
> >wrote:
> >
> >> Just make sure your OS doesn't resolve itself as 127.0.0.1, usual
> >> suspect if you are using ubuntu is to look at /etc/hosts and make sure
> >> your hostname resolves to your IP.
> >>
> >> J-D
> >>
> >> On Tue, Jan 12, 2010 at 4:52 PM, steven zhuang <[email protected]
> >
> >> wrote:
> >> > thanks, Jean,
> >> > I figured out that, in the netstat's output I can see
> >> > 127.0.0.1:60000, I don't know if this means it only listen on
> connection
> >> > request from the same machine.
> >> > About the hbase.master configuration, is there
> anything
> >> I
> >> > can use to replace it?
> >> >
> >> >
> >> > On Wed, Jan 13, 2010 at 1:36 AM, Jean-Daniel Cryans <
> [email protected]
> >> >wrote:
> >> >
> >> >> > 10/01/11 21:16:46 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> >> >> /hbase/master
> >> >> > got 127.0.1.1:60000
> >> >>
> >> >> This means that your master registered itself in Zookeeper as
> >> >> 127.0.0.1, you seem to have a network configuration problem.
> >> >>
> >> >> Also the hbase.master configuration is deprecated and unused.
> >> >>
> >> >> J-D
> >> >>
> >> >> On Tue, Jan 12, 2010 at 6:16 AM, steven zhuang <
> [email protected]
> >> >
> >> >> wrote:
> >> >> > hello, list,
> >> >> >
> >> >> > I am now setting up a HBase cluster using HBase
> version
> >> >> > 0.20.2. But I have met some problems which I googled a lot and got
> no
> >> >> > answer.
> >> >> > Please help me.
> >> >> >
> >> >> > I modified the Hbase-site.xml and copy the whole
> >> directory
> >> >> to
> >> >> > another machine.
> >> >> > Using one as the master, after I started the hbase
> >> server, I
> >> >> > CAN see Hmaster / HQuorumPeer / HRegionServer running on Master
> >> >> > and HQuorumPeer / HRegionServer running on the slave node.
> >> >> > Here is what's weird:
> >> >> > I can enter the hbase shell on master node, but on the
> >> other
> >> >> > region server I cannot execute any command, a "list" command would
> >> cause
> >> >> a
> >> >> > list of exception.
> >> >> >
> >> >> > 10/01/11 21:16:46 DEBUG client.HConnectionManager$ClientZKWatcher:
> Got
> >> >> > ZooKeeper event, state: SyncConnected, type: None, path: null
> >> >> > 10/01/11 21:16:46 DEBUG zookeeper.ZooKeeperWrapper: Read ZNode
> >> >> /hbase/master
> >> >> > got 127.0.1.1:60000
> >> >> > 10/01/11 21:16:46 INFO client.HConnectionManager$TableServers:
> >> getMaster
> >> >> > attempt 0 of 5 failed; retrying after sleep of 2000
> >> >> > java.net.ConnectException: Connection refused
> >> >> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> >> >> > at
> >> >> >
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
> >> >> > at
> >> >> >
> >> >>
> >>
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> >> >> > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
> >> >> >
> >> >> > I can create table in the master node's Hbase shell, but
> >> there
> >> >> > sometime is some exception like:
> >> >> > 10/01/12 06:08:15 DEBUG client.HConnectionManager$TableServers:
> >> >> > locateRegionInMeta attempt 2 of 5 failed; retrying after sleep of
> 2000
> >> >> > org.apache.hadoop.hbase.client.NoServerForRegionException: No
> server
> >> >> address
> >> >> > listed in .META. for region t3,,1263305290760
> >> >> > at
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:668)
> >> >> > at
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:594)
> >> >> > at
> >> >> >
> >> >>
> >>
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
> >> >> >
> >> >> > But after this I can use list to see that the table HAS BEEN
> >> BUILT
> >> >> > inside the hdfs.
> >> >> >
> >> >> > the Hbase-site.xml I used:
> >> >> >
> >> >>
> >>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> >> > <configuration>
> >> >> > <property>
> >> >> > <name>hbase.rootdir</name>
> >> >> > <value>hdfs://sz:8998/hbase</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.cluster.distributed</name>
> >> >> > <value>true</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.master</name>
> >> >> > <value>sz:60000</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.tmp.dir</name>
> >> >> > <value>/home/steven/data/hbase-${user.name}</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.zookeeper.property.dataDir</name>
> >> >> > <value>${hbase.tmp.dir}/zookeeper</value>
> >> >> > </property>
> >> >> >
> >> >> > <!--
> >> >> > The following three properties are used together to create the
> list
> >> of
> >> >> > host:peer_port:leader_port quorum servers for ZooKeeper.
> >> >> > -->
> >> >> > <property>
> >> >> > <name>hbase.zookeeper.quorum</name>
> >> >> > <value>sz,hadoop3</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.zookeeper.peerport</name>
> >> >> > <value>2888</value>
> >> >> > </property>
> >> >> >
> >> >> > <property>
> >> >> > <name>hbase.zookeeper.leaderport</name>
> >> >> > <value>3888</value>
> >> >> > </property>
> >> >> >
> >> >>
> >>
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
> >> >> >
> >> >> >
> >> >> > --
> >> >> > best wishes.
> >> >> > steven
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > best wishes.
> >> > steven
> >> >
> >>
> >
> >
> >
> > --
> > best wishes.
> > steven
> >
>
--
best wishes.
steven
