Hi Isuru,
Big +1 for this effort. Shall we create a JIRA to track this and get it
fixed in next release?
I've noticed few other issues related to Stratos threading behavior.
- New threads are created in certain places [1] which should be avoided as
much as possible.
@Reka: Any idea on why we create new Threads in
ClusterLevelPartitionContext at [1]? This would cause Stratos to go out of
resources at some point right?
- New threads are created in component activators [2]. Although level of
impact is minimum since this is a one-time creating, we should be using the
thread pool allocated at all times.
- Here is a code piece found in CC service component;
private void shutdownExecutorService(String executorServiceId) {
ExecutorService executorService =
StratosThreadPool.getExecutorService(executorServiceId, 1);
if (executorService != null) {
shutdownExecutorService(executorService);
}
}
private void shutdownExecutorService(ExecutorService executorService) {
try {
executorService.shutdownNow();
} catch (Exception e) {
log.warn("An error occurred while shutting down executor service", e);
}
}
It really doesn't make sense to get the thread pool object already created
by passing a thread pool size. We should have a fully flexible API in
StratosThreadPool class in Stratos common component to perform those
operations.
[1]
https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/context/partition/ClusterLevelPartitionContext.java#L116-L121
[2]
https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/internal/AutoscalerServiceComponent.java#L120
Thanks.
On Thu, Dec 3, 2015 at 10:20 AM, Isuru Haththotuwa <isu...@apache.org>
wrote:
> Hi Devs,
>
> Currently what we have is a static set of thread pools:
>
> 1. monitor thread pool
> 2. autoscaler thread pool
> 3. cartridge agent event listener thread pool
> 4. cloud controller context thread pool
> 5. cloud controller thread pool
> 6. cloud controller instance manager thread pool
> 7. cloud controller stats publisher thread pool
> 8. autoscaler stats publisher thread pool
> 9. load balancer thread pool
> 10. mock iaas event listener thread pool
> 11. stratos manager thread pool
>
> This is in addition to the scheduled executor services we are using. Shall
> we optimize the StratosThreadPool implementation to dynamically shrink and
> grow when required? We are currently using fixed thread pools.
>
>
>
> On Sun, Nov 22, 2015 at 12:15 PM, Imesh Gunaratne <im...@apache.org>
> wrote:
>
>> Hi Akila,
>>
>> On Sat, Nov 21, 2015 at 1:20 PM, Akila Ravihansa Perera <
>> raviha...@wso2.com> wrote:
>>
>>>
>>> Any idea why thread pools are created per monitor object? Is it not
>>> possible to share a common thread pool as per the current design?
>>>
>>
>> No, we do not create a thread pool per monitor object, have a look at the
>> code carefully:
>>
>> executorService = StratosThreadPool.getExecutorService(
>> AutoscalerConstants.MONITOR_THREAD_POOL_ID,
>> threadPoolSize);
>>
>> The above statement ask Stratos Thread Pool class to return an executor
>> service with the name AutoscalerConstants.MONITOR_THREAD_POOL_ID:
>>
>> public static ExecutorService getExecutorService(String identifier, int
>> threadPoolSize) {
>> ExecutorService executorService =
>> executorServiceMap.get(identifier);
>> if (executorService == null) {
>> synchronized (executorServiceMapLock) {
>> if (executorService == null) {
>> executorService =
>> Executors.newFixedThreadPool(threadPoolSize);
>> executorServiceMap.put(identifier, executorService);
>> log.info(String.format("Thread pool created: [type]
>> Executor Service [id] %s [size] %d", identifier, threadPoolSize));
>> }
>> }
>> }
>> return executorService;
>> }
>>
>> This will only create an executor service once for a given identifier.
>>
>> Thanks
>>
>>
>>>
>>> Stratos server thread count goes beyond 1500 when deploying 2 or 3 apps.
>>> And it keeps growing proportional to the deployed app count. This is a very
>>> high resource usage.
>>>
>>> [1]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/cluster/ClusterMonitor.java#L108
>>>
>>> [2]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ApplicationMonitor.java#L82
>>>
>>> [3]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/GroupMonitor.java#L88
>>>
>>> [4]
>>> https://github.com/apache/stratos/blob/stratos-4.1.x/components/org.apache.stratos.autoscaler/src/main/java/org/apache/stratos/autoscaler/monitor/component/ParentComponentMonitor.java#L112
>>>
>>> Thanks.
>>>
>>> --
>>> Akila Ravihansa Perera
>>> WSO2 Inc.; http://wso2.com/
>>>
>>> Blog: http://ravihansa3000.blogspot.com
>>>
>>
>>
>>
>> --
>> Imesh Gunaratne
>>
>> Senior Technical Lead, WSO2
>> Committer & PMC Member, Apache Stratos
>>
>> --
>> Thanks and Regards,
>>
>> Isuru H.
>> +94 716 358 048* <http://wso2.com/>*
>>
>>
>> * <http://wso2.com/>*
>>
>>
>>
--
Akila Ravihansa Perera
WSO2 Inc.; http://wso2.com/
Blog: http://ravihansa3000.blogspot.com
