Hi All,
I have the problem in my HBase fully distributed mode with backup master
. The HBase in the fully distributed mode is starting and working properly. The
Active Master is working and My backup master is waiting for the active master
termination. If I kill the active master process, the backup master don't come
to active which remains in the waiting state.
I declared the backup master in the HBase/conf/Backup-masters file.
My backup master log is
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: revision
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: hdfsUser
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: hdfsDate
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: hdfsUrl
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: date
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: hdfsRevision
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: user
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: hdfsVersion
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: url
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: MetricsString
added: version
2011-08-02 16:04:16,416 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2011-08-02 16:04:16,417 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2011-08-02 16:04:16,417 INFO
org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized
2011-08-02 16:04:16,418 DEBUG org.apache.hadoop.hbase.master.HMaster: HMaster
started in backup mode. Stalling until master znode is written.
2011-08-02 16:04:16,418 DEBUG org.apache.hadoop.hbase.master.HMaster: Waiting
for master address ZNode to be written (Also watching cluster state node)
2011-08-02 16:07:16,418 DEBUG org.apache.hadoop.hbase.master.HMaster: Waiting
for master address ZNode to be written (Also watching cluster state node)
2011-08-02 16:10:16,418 DEBUG org.apache.hadoop.hbase.master.HMaster: Waiting
for master address ZNode to be written (Also watching cluster state node)
2011-08-02 16:13:16,418 DEBUG org.apache.hadoop.hbase.master.HMaster: Waiting
for master address ZNode to be written (Also watching cluster state node)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
And the one of my region server log is
2011-08-02 16:21:45,977 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache:
LRU Stats: total=958.41 KB, free=198.43 MB, max=199.36 MB, blocks=1,
accesses=10, hits=9, hitRatio=89.99%%, cachingAccesses=10, cachingHits=9,
cachingHitsRatio=89.99%%, evictions=0, evicted=0, evictedPerRun=NaN
2011-08-02 16:22:40,714 WARN
org.apache.hadoop.hbase.regionserver.HRegionServer: Unable to connect to
master. Retrying. Error was:
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy5.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1445)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:737)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:586)
at java.lang.Thread.run(Thread.java:636)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
1. How can we solve this problem?
2. Is there any extra configuration needed for this backup master configuration
in HBase?
Thanks for your valuable
suggestions ......!
Regards,
Shanmuganathan
