I'd say its a concurrency issue on its own, because we didn't touch ZKConfig, but indeed the timing is suspect.
I opened HBASE-4835 for this. - Andy ----- Original Message ----- > From: Mikhail Bautin <[email protected]> > To: [email protected] > Cc: > Sent: Monday, November 21, 2011 1:48 AM > Subject: regionserver crash due to ZK initialization error > > Hello, > > I just saw this in my five-node, three-regionserver cluster test. The > regionserver crashed with this error. Could this be related to some recent > changes involving ZK? Alternatively, this could be a concurrency issue of > its own. > > 2011-11-21 01:30:15,188 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > <hostname>,60020,1321867814890: Initialization of RS failed. Hence > aborting RS. > java.util.ConcurrentModificationException > at java.util.Hashtable$Enumerator.next(Hashtable.java:1031) > at > org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1042) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:75) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:245) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:144) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:124) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559) > at > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183) > at > org.apache.hadoop.hbase.catalog.CatalogTracker.<init>(CatalogTracker.java:177) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:575) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:534) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:642) > at java.lang.Thread.run(Thread.java:619) > > Thanks, > --Mikhail >
