[
https://issues.apache.org/jira/browse/BROOKLYN-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aled Sage updated BROOKLYN-181:
-------------------------------
Attachment: many-threads.txt
Output from jstack
> OutOfMemoryError: unable to create new native thread (when stopping app)
> ------------------------------------------------------------------------
>
> Key: BROOKLYN-181
> URL: https://issues.apache.org/jira/browse/BROOKLYN-181
> Project: Brooklyn
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Aled Sage
> Attachments: many-threads.txt
>
>
> Using 0.9.0-SNAPSHOT...
> I deployed an app to OpenStack (Bluebox) - it failed due to insufficient
> security groups etc. I terminated the VMs in Bluebox itself, and then I tried
> to call the "Stop" effector on the app in Brooklyn. Eventually Brooklyn ran
> out of threads.
> My app consisted of a riak cluster + web cluster.
> However, the log is messy because I had a couple of other apps that I also
> started/stopped during this time (I think they are unrelated though).
> There are 2045 threads, of which 2012 are of the form
> "brooklyn-execmanager-*".
> 998 of these threads are doing:
> {noformat}
> "brooklyn-execmanager-vvDaziiT-2847" daemon prio=5 tid=0x00007fd4f08aa800
> nid=0xc7d0f waiting for monitor entry [0x0000700063d3f000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.brooklyn.camp.brooklyn.spi.dsl.BrooklynDslDeferredSupplier.get(BrooklynDslDeferredSupplier.java:83)
> - waiting to lock <0x00000007c5940bc8> (a
> org.apache.brooklyn.camp.brooklyn.spi.dsl.methods.DslComponent$AttributeWhenReady)
> at
> org.apache.brooklyn.util.core.task.ValueResolver$2.call(ValueResolver.java:322)
> at
> org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob.call(DynamicSequentialTask.java:342)
> at
> org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> Another 999 threads look like:
> {noformat}
> "brooklyn-execmanager-vvDaziiT-2930" daemon prio=5 tid=0x00007fd4ef16f800
> nid=0x5e07 in Object.wait() [0x0000700001703000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at
> org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob$1.call(DynamicSequentialTask.java:275)
> - locked <0x00000007c64bafa8> (a java.lang.Object)
> at
> org.apache.brooklyn.util.core.task.DynamicSequentialTask$DstJob$1.call(DynamicSequentialTask.java:265)
> at
> org.apache.brooklyn.util.core.task.BasicExecutionManager$SubmissionCallable.call(BasicExecutionManager.java:468)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
> The logging shows the number of threads creeping up, with 30 more threads and
> 30 more active tasks being added every minute, e.g.:
> {noformat}
> 2015-10-12 23:01:14,022 DEBUG o.a.b.c.m.i.BrooklynGarbageCollector
> [brooklyn-gc]: brooklyn gc (after) - using 164 MB / 264 MB memory (20.9 kB
> soft); 2001 threads; storage: {datagrid={size=7, createCount=7},
> refsMapSize=0, lis
> tsMapSize=0}; tasks: 1981 active, 999 unfinished; 1084 remembered, 286554
> total submitted)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
