Created [1] to track the improvements, will do the changes for the new release.
[1]. https://issues.apache.org/jira/browse/STRATOS-1639 On Thu, Dec 3, 2015 at 12:38 PM, Reka Thirunavukkarasu <r...@wso2.com> wrote: > Hi Akila, > > On Thu, Dec 3, 2015 at 11:11 AM, Akila Ravihansa Perera < > raviha...@wso2.com> wrote: > >> Hi Isuru, >> >> Big +1 for this effort. Shall we create a JIRA to track this and get it >> fixed in next release? >> >> I've noticed few other issues related to Stratos threading behavior. >> >> - New threads are created in certain places [1] which should be avoided >> as much as possible. >> >> @Reka: Any idea on why we create new Threads in >> ClusterLevelPartitionContext at [1]? This would cause Stratos to go out of >> resources at some point right? >> > > I think this had been added to maintain few tasks for pending member > watcher, obsolete member watcher and etc in the > ClusterLevelPartitionContext. Those can be converted to scheduled tasks > with executor service. > >> >> - New threads are created in component activators [2]. Although level of >> impact is minimum since this is a one-time creating, we should be using the >> thread pool allocated at all times. >> >> - Here is a code piece found in CC service component; >> >> private void shutdownExecutorService(String executorServiceId) { >> ExecutorService executorService = >> StratosThreadPool.getExecutorService(executorServiceId, 1); >> if (executorService != null) { >> shutdownExecutorService(executorService); >> } >> } >> >> private void shutdownExecutorService(ExecutorService executorService) { >> try { >> executorService.shutdownNow(); >> } catch (Exception e) { >> log.warn("An error occurred while shutting down executor service", e); >> } >> } >> >> >> It really doesn't make sense to get the thread pool object already >> created by passing a thread pool size. We should have a fully flexible API >> in StratosThreadPool class in Stratos common component to perform those >> operations. >> >> [1] >> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/partition/ClusterLevelPartitionContext.java#L116-L121 >> >> [2] >> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/internal/AutoscalerServiceComponent.java#L120 >> >> >> Thanks. >> >> On Thu, Dec 3, 2015 at 10:20 AM, Isuru Haththotuwa <isu...@apache.org> >> wrote: >> >>> Hi Devs, >>> >>> Currently what we have is a static set of thread pools: >>> >>> 1. monitor thread pool >>> 2. autoscaler thread pool >>> 3. cartridge agent event listener thread pool >>> 4. cloud controller context thread pool >>> 5. cloud controller thread pool >>> 6. cloud controller instance manager thread pool >>> 7. cloud controller stats publisher thread pool >>> 8. autoscaler stats publisher thread pool >>> 9. load balancer thread pool >>> 10. mock iaas event listener thread pool >>> 11. stratos manager thread pool >>> >>> This is in addition to the scheduled executor services we are using. >>> Shall we optimize the StratosThreadPool implementation to dynamically >>> shrink and grow when required? We are currently using fixed thread pools. >>> >>> >>> >>> On Sun, Nov 22, 2015 at 12:15 PM, Imesh Gunaratne <im...@apache.org> >>> wrote: >>> >>>> Hi Akila, >>>> >>>> On Sat, Nov 21, 2015 at 1:20 PM, Akila Ravihansa Perera < >>>> raviha...@wso2.com> wrote: >>>> >>>>> >>>>> Any idea why thread pools are created per monitor object? Is it not >>>>> possible to share a common thread pool as per the current design? >>>>> >>>> >>>> No, we do not create a thread pool per monitor object, have a look at >>>> the code carefully: >>>> >>>> executorService = StratosThreadPool.getExecutorService( >>>> AutoscalerConstants.MONITOR_THREAD_POOL_ID, >>>> threadPoolSize); >>>> >>>> The above statement ask Stratos Thread Pool class to return an executor >>>> service with the name AutoscalerConstants.MONITOR_THREAD_POOL_ID: >>>> >>>> public static ExecutorService getExecutorService(String identifier, int >>>> threadPoolSize) { >>>> ExecutorService executorService = >>>> executorServiceMap.get(identifier); >>>> if (executorService == null) { >>>> synchronized (executorServiceMapLock) { >>>> if (executorService == null) { >>>> executorService = >>>> Executors.newFixedThreadPool(threadPoolSize); >>>> executorServiceMap.put(identifier, executorService); >>>> log.info(String.format("Thread pool created: >>>> [type] Executor Service [id] %s [size] %d", identifier, threadPoolSize)); >>>> } >>>> } >>>> } >>>> return executorService; >>>> } >>>> >>>> This will only create an executor service once for a given identifier. >>>> >>>> Thanks >>>> >>>> >>>>> >>>>> Stratos server thread count goes beyond 1500 when deploying 2 or 3 >>>>> apps. And it keeps growing proportional to the deployed app count. This is >>>>> a very high resource usage. >>>>> >>>>> [1] >>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/cluster/ClusterMonitor.java#L108 >>>>> >>>>> [2] >>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ApplicationMonitor.java#L82 >>>>> >>>>> [3] >>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/GroupMonitor.java#L88 >>>>> >>>>> [4] >>>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ParentComponentMonitor.java#L112 >>>>> >>>>> Thanks. >>>>> >>>>> -- >>>>> Akila Ravihansa Perera >>>>> WSO2 Inc.; http://wso2.com/ >>>>> >>>>> Blog: http://ravihansa3000.blogspot.com >>>>> >>>> >>>> >>>> >>>> -- >>>> Imesh Gunaratne >>>> >>>> Senior Technical Lead, WSO2 >>>> Committer & PMC Member, Apache Stratos >>>> >>>> -- >>>> Thanks and Regards, >>>> >>>> Isuru H. >>>> +94 716 358 048* <http://wso2.com/>* >>>> >>>> >>>> * <http://wso2.com/>* >>>> >>>> >>>> >> >> >> -- >> Akila Ravihansa Perera >> WSO2 Inc.; http://wso2.com/ >> >> Blog: http://ravihansa3000.blogspot.com >> > > > > -- > Reka Thirunavukkarasu > Senior Software Engineer, > WSO2, Inc.:http://wso2.com, > Mobile: +94776442007 > > -- > <%2B94776442007> > <%2B94776442007> > Thanks and Regards, > > Isuru H. > <%2B94776442007> > +94 716 358 048 <%2B94776442007>* <http://wso2.com/>* > > > * <http://wso2.com/>* > > >
