[ https://issues.apache.org/jira/browse/HBASE-4835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13154790#comment-13154790 ]
stack commented on HBASE-4835: ------------------------------ There's a few per JVM: {code} src/main/java/org/apache/hadoop/hbase/master/HMaster.java: this.zooKeeper = new ZooKeeperWatcher(conf, MASTER + ":" + isa.getPort(), this, true); src/main/java/org/apache/hadoop/hbase/master/HMaster.java: this.zooKeeper = new ZooKeeperWatcher(conf, MASTER + ":" src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java: this.zooKeeper = new ZooKeeperWatcher(conf, REGIONSERVER + ":" + src/main/java/org/apache/hadoop/hbase/replication/master/ReplicationLogCleaner.java: new ZooKeeperWatcher(this.conf, "replicationLogCleaner", null); src/main/java/org/apache/hadoop/hbase/replication/ReplicationPeer.java: zkw = new ZooKeeperWatcher(conf, src/main/java/org/apache/hadoop/hbase/catalog/CatalogTracker.java: new ZooKeeperWatcher(conf, "catalogtracker-on-" + connection.toString(), src/main/java/org/apache/hadoop/hbase/client/HConnectionManager.java: this.zooKeeper = new ZooKeeperWatcher(conf, "hconnection", this); {code} When you say: bq. The concern I have about cloning in makeZKProps is the same hashmap iteration will happen for that. You are afraid that the clone is not a deep clone so we'll just be iterating the same map of configuration items? That seems valid enough though why won't we have same issue if we make a clone in ZKW? I'm wary cloning Configuration. I did that 'fixing' our issues w/ tests where we were sharing an HConnection and I manufactured the issue where we had too many connections against the zk ensemble. Odd we see this now. > ConcurrentModificationException out of ZKConfig.makeZKProps > ----------------------------------------------------------- > > Key: HBASE-4835 > URL: https://issues.apache.org/jira/browse/HBASE-4835 > Project: HBase > Issue Type: Bug > Affects Versions: 0.92.0, 0.94.0 > Reporter: Andrew Purtell > Assignee: Andrew Purtell > Attachments: HBASE-4835.patch > > > Mikhail reported this from a five-node, three-RS cluster test: > {code} > 2011-11-21 01:30:15,188 FATAL > org.apache.hadoop.hbase.regionserver.HRegionServer: ABORTING region server > <machine_name>,60020,1321867814890: Initialization of RS failed. Hence > aborting RS. > java.util.ConcurrentModificationException > at java.util.Hashtable$Enumerator.next(Hashtable.java:1031) > at org.apache.hadoop.conf.Configuration.iterator(Configuration.java:1042) > at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps(ZKConfig.java:75) > at > org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString(ZKConfig.java:245) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:144) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:124) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1262) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:568) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:559) > at > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:183) > at > org.apache.hadoop.hbase.catalog.CatalogTracker.<init>(CatalogTracker.java:177) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.initializeZooKeeper(HRegionServer.java:575) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:534) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:642) > at java.lang.Thread.run(Thread.java:619) > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira