[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093470#comment-13093470 ] Hadoop QA commented on ZOOKEEPER-1140: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490942/ZOOKEEPER-1140.patch against trunk revision 1163015. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/478//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/478//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/478//console This message is automatically generated. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Attachments: ZOOKEEPER-1140.patch Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093560#comment-13093560 ] Laxman commented on ZOOKEEPER-1140: --- Thanks for review and commit Mahadev. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Attachments: ZOOKEEPER-1140.patch Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093647#comment-13093647 ] Hudson commented on ZOOKEEPER-1140: --- Integrated in ZooKeeper-trunk #1288 (See [https://builds.apache.org/job/ZooKeeper-trunk/1288/]) ZOOKEEPER-1140. server shutdown is not stopping threads. (laxman via mahadev) mahadev : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1163102 Files : * /zookeeper/trunk/CHANGES.txt * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/LearnerHandler.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/QuorumPeer.java * /zookeeper/trunk/src/java/main/org/apache/zookeeper/server/quorum/ReadOnlyZooKeeperServer.java server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Attachments: ZOOKEEPER-1140.patch Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13093456#comment-13093456 ] Mahadev konar commented on ZOOKEEPER-1140: -- +1 looks good to me. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Attachments: ZOOKEEPER-1140.patch Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13087666#comment-13087666 ] Hadoop QA commented on ZOOKEEPER-1140: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490942/ZOOKEEPER-1140.patch against trunk revision 1159432. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/468//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/468//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-ZOOKEEPER-Build/468//console This message is automatically generated. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Attachments: ZOOKEEPER-1140.patch Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13082253#comment-13082253 ] Laxman commented on ZOOKEEPER-1140: --- Two leaks has been identified. 1) SendThread is getting leaked. [LearnerHandler] *Approach to fix* a) Add the proposalOfDeath in shutdown() rather in finally block. 2) ROZK is getting shutdown even before startup. *Approach to fix* a) Synchronize the startup and shutdown of ROZK to avoid concurrent startp and shutdown. b) Add a flag shutdown and check in startup to avoid startup after shutdown. c) Interrupt the thread that is starting ROZK and join on that thread in finally block before calling shutdown. Both of these leaks I'm planning to address in my patch as per a/m approach. Please let me know if you have any different approach or any suggestion. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Assignee: Laxman Priority: Blocker Fix For: 3.4.0 Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13082254#comment-13082254
]
Laxman commented on ZOOKEEPER-1140:
---
{quote}Shouldn't R/O support only be enabled via an explicit configuration
option? i.e. off by default.{quote}
I feel we may need to address the above comment in separate jira.
server shutdown is not stopping threads
---
Key: ZOOKEEPER-1140
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140
Project: ZooKeeper
Issue Type: Bug
Components: server, tests
Affects Versions: 3.4.0
Reporter: Patrick Hunt
Assignee: Laxman
Priority: Blocker
Fix For: 3.4.0
Near the end of QuorumZxidSyncTest there are tons of threads running - 115
ProcessThread threads, similar numbers of SessionTracker.
Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate
thread? (henry/flavio?)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13082146#comment-13082146
]
Laxman commented on ZOOKEEPER-1140:
---
Thread [Sender-/127.0.0.1:3084] (Running)
When I observed QuorumZxidSyncTest, the above thread is also leaking in a
different scenario apart from ReadOnlyZKServer. Above mentioned thread will be
shutdown only if it receives proposalOfDeath.
We are queuing proposalOfDeath in finally block of LearnerHandler.run().
To summarize, proposalOfDeath may not queued up when LearnerHandler receives
IOException and other thread calling LearnerHandler.shutdown(). This leads to
failure of queuing the proposalOfDeath.
To solve this, can we add the proposalOfDeath in shutdown() rather in finally
block.
Anyways, this finding will solve one of the leaks. We still need to fix other
leaks caused by ROZK.
{quote}Just to see if I understand now, are you saying that the test is
spawning over one hundred servers because we are shutting before the server
actually starts? If so, then it is certainly a problem.{quote}
Yes. I will add more analysis and possibly patch today.
server shutdown is not stopping threads
---
Key: ZOOKEEPER-1140
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140
Project: ZooKeeper
Issue Type: Bug
Components: server, tests
Affects Versions: 3.4.0
Reporter: Patrick Hunt
Priority: Blocker
Fix For: 3.4.0
Near the end of QuorumZxidSyncTest there are tons of threads running - 115
ProcessThread threads, similar numbers of SessionTracker.
Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate
thread? (henry/flavio?)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[
https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13073560#comment-13073560
]
Laxman commented on ZOOKEEPER-1140:
---
Thanks for the clarifications Flavio.
I agree with your explanation in #1 and #2.
{quote}shutdown() should have no effect if the zookeeper server hasn't started.
Are you concerned that we might end up creating RO zookeeper servers and using
them unnecessarily?
{quote}
server shutdown is not stopping threads
---
Key: ZOOKEEPER-1140
URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140
Project: ZooKeeper
Issue Type: Bug
Components: server, tests
Affects Versions: 3.4.0
Reporter: Patrick Hunt
Priority: Blocker
Fix For: 3.4.0
Near the end of QuorumZxidSyncTest there are tons of threads running - 115
ProcessThread threads, similar numbers of SessionTracker.
Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate
thread? (henry/flavio?)
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072912#comment-13072912 ] Patrick Hunt commented on ZOOKEEPER-1140: - Comment on list from lakshman_ch: In QuorumPeer, when the peer is in LOOKING state we are starting ReadOnlyZooKeeperServer in a separate thread. And we are shutting down this server even before startup which has no effect. Also, as this is not a blocking call QP keeps on spawning new servers. 1) ReadOnlyZooKeeperServer.startup() need not be called in separate a thread. 2) ReadOnlyZooKeeperServer.startup() is not a blocking call. Need to introduce a method like Leader.lead(), Follower.followLeader() 3) Shutdown should be called only after the a/m blocking call is returned. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Priority: Blocker Fix For: 3.4.0 Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (ZOOKEEPER-1140) server shutdown is not stopping threads
[ https://issues.apache.org/jira/browse/ZOOKEEPER-1140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13072914#comment-13072914 ] Patrick Hunt commented on ZOOKEEPER-1140: - Shouldn't R/O support only be enabled via an explicit configuration option? i.e. off by default. server shutdown is not stopping threads --- Key: ZOOKEEPER-1140 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-1140 Project: ZooKeeper Issue Type: Bug Components: server, tests Affects Versions: 3.4.0 Reporter: Patrick Hunt Priority: Blocker Fix For: 3.4.0 Near the end of QuorumZxidSyncTest there are tons of threads running - 115 ProcessThread threads, similar numbers of SessionTracker. Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
