[jira] [Updated] (AMBARI-14671) Reevaluate the use of threadpools in Ambari code base
[
https://issues.apache.org/jira/browse/AMBARI-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayush Luniya updated AMBARI-14671:
---
Fix Version/s: (was: 2.5.0)
3.0.0
> Reevaluate the use of threadpools in Ambari code base
> -
>
> Key: AMBARI-14671
> URL: https://issues.apache.org/jira/browse/AMBARI-14671
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.2.0
>Reporter: Jayush Luniya
>Assignee: Jayush Luniya
> Fix For: 3.0.0
>
> Attachments: DynamicScaling.tgz
>
>
> As part of investigation for BUG-43981, noticed that in many places in the
> Ambari code base, the way we use the threadpool is not quite correct. We
> will never scale up the number of threads on high load when we use unbounded
> queues. This could lead to performance bottlenecks especially if we are
> configure ThreadPoolExecutor with corePoolSize=0 and maxPoolSize=10, only one
> thread will ever be spawned. See observations below
> Observations:
> 1. When a ThreadPoolExecutor object is created, the pool size is 0 (i.e. no
> new threads are created then) unless prestartAllCoreThreads() is called. Also
> if we set allowCoreThreadTimeOut(true), idle core threads will also be
> reclaimed.
> 2. In our code base, I observed that we create a threadpool using unlimited
> queue. However the pool will never scale up from coreThreads -> maxThreads as
> the request will always get queued.
> {code:java}
> LinkedBlockingQueue queue = new LinkedBlockingQueue(); //
> unlimited Queue
> ThreadPoolExecutor threadPoolExecutor =
> new ThreadPoolExecutor(
> THREAD_POOL_CORE_SIZE,
> THREAD_POOL_MAX_SIZE,
> THREAD_POOL_TIMEOUT_MILLIS,
> TimeUnit.MILLISECONDS,
> queue);
> {code}
> http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
> {quote}
> Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue
> without a predefined capacity) will cause new tasks to wait in the queue when
> all corePoolSize threads are busy. Thus, no more than corePoolSize threads
> will ever be created. (And the value of the maximumPoolSize therefore doesn't
> have any effect.) This may be appropriate when each task is completely
> independent of others, so tasks cannot affect each others execution; for
> example, in a web page server. While this style of queuing can be useful in
> smoothing out transient bursts of requests, it admits the possibility of
> unbounded work queue growth when commands continue to arrive on average
> faster than they can be processed.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AMBARI-14671) Reevaluate the use of threadpools in Ambari code base
[
https://issues.apache.org/jira/browse/AMBARI-14671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayush Luniya updated AMBARI-14671:
---
Fix Version/s: (was: 2.4.0)
2.5.0
> Reevaluate the use of threadpools in Ambari code base
> -
>
> Key: AMBARI-14671
> URL: https://issues.apache.org/jira/browse/AMBARI-14671
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
>Affects Versions: 2.2.0
>Reporter: Jayush Luniya
>Assignee: Jayush Luniya
> Fix For: 2.5.0
>
> Attachments: DynamicScaling.tgz
>
>
> As part of investigation for BUG-43981, noticed that in many places in the
> Ambari code base, the way we use the threadpool is not quite correct. We
> will never scale up the number of threads on high load when we use unbounded
> queues. This could lead to performance bottlenecks especially if we are
> configure ThreadPoolExecutor with corePoolSize=0 and maxPoolSize=10, only one
> thread will ever be spawned. See observations below
> Observations:
> 1. When a ThreadPoolExecutor object is created, the pool size is 0 (i.e. no
> new threads are created then) unless prestartAllCoreThreads() is called. Also
> if we set allowCoreThreadTimeOut(true), idle core threads will also be
> reclaimed.
> 2. In our code base, I observed that we create a threadpool using unlimited
> queue. However the pool will never scale up from coreThreads -> maxThreads as
> the request will always get queued.
> {code:java}
> LinkedBlockingQueue queue = new LinkedBlockingQueue(); //
> unlimited Queue
> ThreadPoolExecutor threadPoolExecutor =
> new ThreadPoolExecutor(
> THREAD_POOL_CORE_SIZE,
> THREAD_POOL_MAX_SIZE,
> THREAD_POOL_TIMEOUT_MILLIS,
> TimeUnit.MILLISECONDS,
> queue);
> {code}
> http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html
> {quote}
> Unbounded queues. Using an unbounded queue (for example a LinkedBlockingQueue
> without a predefined capacity) will cause new tasks to wait in the queue when
> all corePoolSize threads are busy. Thus, no more than corePoolSize threads
> will ever be created. (And the value of the maximumPoolSize therefore doesn't
> have any effect.) This may be appropriate when each task is completely
> independent of others, so tasks cannot affect each others execution; for
> example, in a web page server. While this style of queuing can be useful in
> smoothing out transient bursts of requests, it admits the possibility of
> unbounded work queue growth when commands continue to arrive on average
> faster than they can be processed.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
