[ https://issues.apache.org/jira/browse/ZOOKEEPER-1214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13572143#comment-13572143 ]
Edward Ribeiro commented on ZOOKEEPER-1214: ------------------------------------------- https://issues.apache.org/jira/browse/ZOOKEEPER-1214 Hi Michi, My two cents: this looks like a bug in ReentrantReadWriteLock, the JVM, or even an incorrect use of _ReentrantReadWriteLock_ by JmxMBeanServer. I say so because a quick Google search shows that other projects that use ReentrantReadWriteLock, *directly or not*, as HBase, Netbeans and Maven have hit this very same exception. As ZooKeeper uses _ReentrantReadWriteLock_ indirectly through JmxMBeanServer, a possible solution seems to be to create a lock object and synchronize lines 98 and 117 of JmxMBeanServer as follows: {code:title=JmxMBeanServer.java|borderStyle=solid} ... private final Object LOCK = new Object(); ... // line 98 becomes synchronize (LOCK) { mBeanServer.registerMBean(bean, oname); } // line 117 becomes synchronize (LOCK) { mBeanServer.unregisterMBean(makeObjectName(path, bean)); } {code} As this is a issue addressed to Cesar I'll not submit a patch, but I may provide you a patch by email to see if this solution works.;) Cheers, Edward > QuorumPeer should unregister only its previsously registered MBeans instead > of use MBeanRegistry.unregisterAll() method. > ------------------------------------------------------------------------------------------------------------------------ > > Key: ZOOKEEPER-1214 > URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1214 > Project: ZooKeeper > Issue Type: Bug > Components: quorum > Reporter: César Álvarez Núñez > Assignee: César Álvarez Núñez > Fix For: 3.5.0 > > Attachments: ZOOKEEPER-1214.patch, ZOOKEEPER-1214.patch, > ZOOKEEPER-1214.patch > > > When a QuorumPeer thread dies, it is unregistering *all* ZKMBeanInfo MBeans > previously registered on its java process; including those that has not been > registered by itself. > It does not cause any side effect in production environment where each server > is running on a separate java process; but fails when using > "org.apache.zookeeper.test.QuorumUtil" to programmatically start up a > zookeeper server ensemble and use its provided methods to force Disconnected, > SyncConnected or SessionExpired events; in order to perform some > basic/functional testing. > Scenario: > * QuorumUtil qU = new QuorumUtil(1); // It creates a 3 servers ensemble. > * qU.startAll(); // Startup all servers: 1 Leader + 2 Followers > * qU.shutdown\(i\); // i is a number from 1 to 3. It shutdown one server. > The last method causes that a QuorumPeer will die, invoking the > MBeanRegistry.unregisterAll() method. > As a result, *all* ZKMBeanInfo MBeans are unregistered; including those > belonging to the other QuorumPeer instances. > When trying to restart previous server (qU.restart\(i\)) an AssertionError is > thrown at MBeanRegistry.register(ZKMBeanInfo bean, ZKMBeanInfo parent) > method, causing the QuorumPeer thread dead. > To solve it: > * MBeanRegistry.unregisterAll() method has been removed. > * QuorumPeer only unregister its ZKMBeanInfo MBeans. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira