[ 
https://issues.apache.org/jira/browse/STRATOS-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Nguyen updated STRATOS-812:
-----------------------------------
    Description: 
We have observed this issue on multiple testbeds where we observed VMs were 
being re-spawn after Stratos has been running for about 7-9 days. Wso2carbon 
log shows numerous OutOfMemoryError. Memory dump was auto-generated when this 
happened. Please find in the attached file (debug_info_20140922.tar.gz) the 
following items: Stratos logs, thread dump, screenshot showing 3 memory leak 
suspects.  Cannot attach memory dump file here due to the size limit 
restriction.

TID: [0] [STRATOS] [2014-09-07 03:43:48,708] ERROR 
{org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - GC 
overhead limit exceeded 
{org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} 
java.lang.OutOfMemoryError: GC overhead limit exceeded 
        at java.util.concurrent.Executors.callable(Executors.java:369) 
        at java.util.concurrent.FutureTask.<init>(FutureTask.java:152) 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.<init>(ScheduledThreadPoolExecutor.java:207)
 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:527)
 
        at 
org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor.run(SecondDerivativeFinderWindowProcessor.java:156)
 
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745) 

  was:
We have observed this issue on multiple testbeds where we observed VMs were 
being re-spawn after Stratos has been running for about 7-9 days. Wso2carbon 
log shows numerous OutOfMemoryError. Memory dump was auto-generated when this 
happened. Please find in the attached file (debug_info_20140922.tar.gz) the 
following items: Stratos logs, memory dump, thread dump, screenshot showing 3 
memory leak suspects. 

TID: [0] [STRATOS] [2014-09-07 03:43:48,708] ERROR 
{org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - GC 
overhead limit exceeded 
{org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} 
java.lang.OutOfMemoryError: GC overhead limit exceeded 
        at java.util.concurrent.Executors.callable(Executors.java:369) 
        at java.util.concurrent.FutureTask.<init>(FutureTask.java:152) 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.<init>(ScheduledThreadPoolExecutor.java:207)
 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:527)
 
        at 
org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor.run(SecondDerivativeFinderWindowProcessor.java:156)
 
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 
        at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
        at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
        at java.lang.Thread.run(Thread.java:745) 


> Longetivity test issue: VMs spawned and Stratos threw OutOfMemoryError
> ----------------------------------------------------------------------
>
>                 Key: STRATOS-812
>                 URL: https://issues.apache.org/jira/browse/STRATOS-812
>             Project: Stratos
>          Issue Type: Bug
>    Affects Versions: 4.0.0
>         Environment: Openstack, Stratos 4.0.0
>            Reporter: Jeffrey Nguyen
>              Labels: performance
>         Attachments: debug_info_20140922.tar.gz
>
>
> We have observed this issue on multiple testbeds where we observed VMs were 
> being re-spawn after Stratos has been running for about 7-9 days. Wso2carbon 
> log shows numerous OutOfMemoryError. Memory dump was auto-generated when this 
> happened. Please find in the attached file (debug_info_20140922.tar.gz) the 
> following items: Stratos logs, thread dump, screenshot showing 3 memory leak 
> suspects.  Cannot attach memory dump file here due to the size limit 
> restriction.
> TID: [0] [STRATOS] [2014-09-07 03:43:48,708] ERROR 
> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} - GC 
> overhead limit exceeded 
> {org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor} 
> java.lang.OutOfMemoryError: GC overhead limit exceeded 
>         at java.util.concurrent.Executors.callable(Executors.java:369) 
>         at java.util.concurrent.FutureTask.<init>(FutureTask.java:152) 
>         at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.<init>(ScheduledThreadPoolExecutor.java:207)
>  
>         at 
> java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:527)
>  
>         at 
> org.apache.stratos.cep.extension.SecondDerivativeFinderWindowProcessor.run(SecondDerivativeFinderWindowProcessor.java:156)
>  
>         at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
>         at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
>  
>         at 
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
>  
>         at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>  
>         at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  
>         at java.lang.Thread.run(Thread.java:745) 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to