Near the end of this test (QuorumZxidSyncTest) there are tons of threads running - 115 "ProcessThread" threads, similar numbers of SessionTracker.
Also I see ~100 ReadOnlyRequestProcessor - why is this running as a separate thread? (henry/flavio?) Regardless, I'll enter a 3.4.0 blocker to clean this up - I suspect that the server shutdown is not shutting down fully for some reason. Patrick On Thu, Jul 28, 2011 at 5:28 PM, Mahadev Konar <maha...@hortonworks.com> wrote: > Nice find Pat. I cant see a reason on why that should happen. Can we > just do a stack dump and compare? > > thanks > mahadev > > On Thu, Jul 28, 2011 at 1:54 PM, Patrick Hunt <ph...@apache.org> wrote: >> I tracked this down to a low ulimit setting on the particular jenkins >> host where this was failing (max processes). >> >> Specifically the following test was failing on trunk, but not on >> branch 3_3, which concerns me >> ./src/java/test/org/apache/zookeeper/test/QuorumZxidSyncTest.java >> >> there haven't been any real changes to this test between versions, any >> insight into why the server is using more threads in trunk vs >> branch33? >> >> Patrick >> >> On Fri, Jul 22, 2011 at 10:58 AM, Patrick Hunt <ph...@apache.org> wrote: >>> I've never seen this before, but in my CI environment (sun jdk >>> 1.6.0_20) I'm seeing some intermittent failures such as the following. >>> >>> Has anyone added/modified tests for 3.4.0 that might be using more >>> threads/memory than previously? Creating ZK clients but not closing >>> them, etc... >>> >>> java.lang.OutOfMemoryError: unable to create new native thread >>> at java.lang.Thread.start0(Native Method) >>> at java.lang.Thread.start(Thread.java:597) >>> at >>> org.apache.zookeeper.server.NIOServerCnxnFactory.start(NIOServerCnxnFactory.java:114) >>> at >>> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:406) >>> at >>> org.apache.zookeeper.test.QuorumBase.startServers(QuorumBase.java:186) >>> at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:103) >>> at org.apache.zookeeper.test.QuorumBase.setUp(QuorumBase.java:67) >>> at >>> org.apache.zookeeper.test.QuorumZxidSyncTest.setUp(QuorumZxidSyncTest.java:39) >>> >>> Patrick >>> >> >