If you still cannot move out of 0.90.0 try to understand the code in HRegionServer.reportForDuty() in 0.90.3 and merge it in your copy of 0.90.0 and try. Just a suggestion :)
Regards Ram -----Original Message----- From: V.Sriram [mailto:srira...@ask.com] Sent: Thursday, November 03, 2011 1:11 PM To: hbase-u...@hadoop.apache.org Subject: RE: RE: Hbase Multiple Master issue Thats great news. Thanks for ur valuable info!! I will use that =) Stuti Awasthi wrote: > > Hi Sriram, > I am using Hbase 0.90.3 and I also tested Multiple master issue. It > worked fine for my cluster. Once new Master is up, after sometime > region server start communicating with new Master. > Hope this helps. > > -----Original Message----- > From: V.Sriram [mailto:srira...@ask.com] > Sent: Thursday, November 03, 2011 12:54 PM > To: hbase-u...@hadoop.apache.org > Subject: Re: RE: Hbase Multiple Master issue > > > I saw the release notes of Hbase 0.90.2, 0.90.3 and 0.90.4. This issue > doesn't seem to be mentioned there. Also since our team want continue > with 0.90.0 itself for some more time, I am not currently upgrading to > newer versions. > > Today saw the Zk_dump to verify if the new master is currently > reflected in Zookeeper. It is correctly reflected. But even then the > region servers don't communicate with the new master. > > I think region servers would ideally communicate with zookeeper and > get the new master node and then communicate with it. But thats not > happenning!!! > Any suggestions plz !! > > > V.Sriram wrote: >> >> Thanks for the reply. Will try other versions as well!! >> >> regards, >> V.Sriram >> >> >> Ramkrishna S Vasudevan wrote: >>> >>> Hi >>> >>> If i remember correctly this issue was fixed in later versions of >>> 0.90.0. >>> Could you try this in other versions and see if this problem >>> persists..ideally it should not. >>> >>> >>> Regards >>> Ram >>> >>> >>> >>> ----- Original Message ----- >>> From: "V.Sriram" <srira...@ask.com> >>> Date: Tuesday, November 1, 2011 3:01 pm >>> Subject: RE: Hbase Multiple Master issue >>> To: hbase-u...@hadoop.apache.org >>> >>>> >>>> I am using 0.90.0 version >>>> >>>> regards, >>>> V.Sriram >>>> >>>> >>>> Ramkrishna S Vasudevan wrote: >>>> > >>>> > Hi >>>> > >>>> > What version of HBase are you using? >>>> > >>>> > Regards >>>> > Ram >>>> > >>>> > >>>> > >>>> > -----Original Message----- >>>> > From: V.Sriram [mailto:srira...@ask.com] >>>> > Sent: Tuesday, November 01, 2011 12:58 PM >>>> > To: hbase-u...@hadoop.apache.org >>>> > Subject: Hbase Multiple Master issue >>>> > >>>> > >>>> > Hi all, >>>> > >>>> > I am evaluating HBase Multiple master utility as I require that >>>> > funtionality. I used a simple 4 machine hbase cluster with M1 as >>>> master.> Now >>>> > I started the Hmaster in M4 as well. I could see logs in M4 >>>> Master log as, >>>> > This HMaster is waiting as already M1 is acting as HMaster. >>>> > >>>> > After this I stopped the Master in the M1 and after few seconds, >>>> the M4 >>>> > master took over. But I am not seeing any region server >>>> communicating to >>>> > this new master. The region server logs show, >>>> > >>>> > 2011-10-31 23:21:18,834 WARN >>>> > org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to >>>> connect to >>>> > master. Retrying. Error was: >>>> > java.net.ConnectException: Connection refused >>>> > at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) >>>> > at >>>> > >>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:5 >>>> 74)> >>>> at >>>> > >>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeo >>>> u >>>> t.java:2> >>>> 06) >>>> > at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(H >>>> B >>>> aseClien> >>>> t.java:311) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.j >>>> ava:865)> >>>> at >>>> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:2 >>>> 57)> at $Proxy3.getProtocolVersion(Unknown Source) >>>> > at >>>> > org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419) >>>> > at >>>> > org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393) >>>> > at >>>> > org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444) >>>> > at >>>> > org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegio >>>> n >>>> Server.j> >>>> ava:1443) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerR >>>> e >>>> port(HRe> >>>> gionServer.java:737) >>>> > at >>>> > >>>> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServe >>>> r >>>> .java:58> >>>> 6) >>>> > at java.lang.Thread.run(Thread.java:619) >>>> > >>>> > Also in the M4 Master log I am seeing, >>>> > >>>> > 2011-10-31 23:20:47,671 INFO org.apache.hadoop.http.HttpServer: >>>> > listener.getLocalPort() returned 60010 >>>> > webServer.getConnectors()[0].getLocalPort() returned 60010 >>>> > 2011-10-31 23:20:47,671 INFO org.apache.hadoop.http.HttpServer: >>>> Jetty> bound >>>> > to port 60010 >>>> > 2011-10-31 23:20:47,671 INFO org.mortbay.log: jetty-6.1.26 >>>> > 2011-10-31 23:20:48,691 INFO org.mortbay.log: Started >>>> > SelectChannelConnector@0.0.0.0:60010 >>>> > 2011-10-31 23:20:48,691 DEBUG >>>> org.apache.hadoop.hbase.master.HMaster:> Started service threads >>>> > 2011-10-31 23:20:50,192 INFO >>>> org.apache.hadoop.hbase.master.ServerManager:> Waiting on >>>> regionserver(s) to checkin >>>> > 2011-10-31 23:20:51,695 INFO >>>> org.apache.hadoop.hbase.master.ServerManager:> Waiting on >>>> regionserver(s) to checkin >>>> > 2011-10-31 23:20:53,197 INFO >>>> org.apache.hadoop.hbase.master.ServerManager:> Waiting on >>>> regionserver(s) to checkin >>>> > >>>> > Seems the region servers are only communicating to the M1 Hmaster >>>> and not >>>> > to >>>> > the new one!! >>>> > >>>> > Any help regarding this is highly appreciated :handshake: >>>> > -- >>>> > View this message in context: >>>> > http://old.nabble.com/Hbase-Multiple-Master-issue- >>>> tp32757197p32757197.html> Sent from the HBase User mailing list >>>> archive at Nabble.com. >>>> > >>>> > >>>> > >>>> >>>> -- >>>> View this message in context: http://old.nabble.com/Hbase-Multiple- >>>> Master-issue-tp32757197p32757528.html >>>> Sent from the HBase User mailing list archive at Nabble.com. >>>> >>>> >>> >>> >> >> > > -- > View this message in context: > http://old.nabble.com/Hbase-Multiple-Master-issue-tp32757197p32770898. > html Sent from the HBase User mailing list archive at Nabble.com. > > > ::DISCLAIMER:: > ---------------------------------------------------------------------- > ------------------------------------------------- > > The contents of this e-mail and any attachment(s) are confidential and > intended for the named recipient(s) only. > It shall not attach any liability on the originator or HCL or its > affiliates. Any views or opinions presented in this email are solely > those of the author and may not necessarily reflect the opinions of > HCL or its affiliates. > Any form of reproduction, dissemination, copying, disclosure, > modification, distribution and / or publication of this message > without the prior written consent of the author of this e-mail is > strictly prohibited. If you have received this email in error please > delete it and notify the sender immediately. Before opening any mail > and attachments please check them for viruses and defect. > > ---------------------------------------------------------------------- > ------------------------------------------------- > > -- View this message in context: http://old.nabble.com/Hbase-Multiple-Master-issue-tp32757197p32770950.html Sent from the HBase User mailing list archive at Nabble.com.
