I like this approach, it sounds elegant.
Cheers,
Peter.
Patricia Shanahan wrote:
I've thought of a significant simplification, based on the idea that I
only need samples of the task count.
I was going to try to use a concurrent queue and deal with observations
getting out of order. I just realized I can make it much cleaner and
simpler by using a ReentrantLock. The manager thread would use
lockInterruptibly, and wait indefinitely to get the lock if another
thread holds it.
A thread that has just made a change to the task count would use
tryLock(). If it gets the lock, it can add its report to a simple
LinkedList. If it fails to get the lock it would just carry on without
collecting a sample. Either the management thread is processing
collected data, or another thread is logging the task count at about the
same time. Either way it does not need to collect a sample.
This strategy automatically throttles the number of observations that
get collected for the manager to consider, though I may also want a way
to further limit the number of messages. However, that can be done more
simply, and with better information, inside code controlled by the lock.
Patricia
On 8/9/2011 12:36 PM, Patricia Shanahan wrote:
I'm investigating extended ThreadPoolExecutor to get an Executor that
bases the number of threads on the mean number of tasks over an
interval. If successful, that would flatten out sudden peaks in the task
load, using the queue as a buffer, but would make the number of threads
match a sustained load, so that there would be no unnecessary
bottleneck.
The current TaskManager allows specification of e.g. one thread per
three tasks, which does result in a lower rate of change but can leave
the number of threads a fraction of the number of tasks in a sustained
workload.
Here is a general outline of a design:
Methods that change the number of tasks would log the new number of
tasks and a timestamp on a concurrent queue. As a refinement, there may
be an option to use a ThreadLocal Random to log only with a specified
probability to bound the number of messages on the queue.
A thread (more about which thread later) would periodically drain the
queue into a local TreeSet, getting them sorted, and then calculate the
mean number of tasks and adjust the thread count accordingly.
The big design problem is which thread does this work. Here are the
options I'm considering:
1. Add it to the tasks, so that a ThreadPoolExecutor worker thread does
it. The downside is that all the existing worker threads could get busy
on long running tasks and tasks are building up in the queue. Nothing
would be done about it until a worker thread finishes its current task.
2. Do it from the execute method, in whatever thread is adding the task.
Again, we could get into situations in which nothing is done about a
situation that could be improved by adding threads. The effect would be
to make the execute method randomly slow from the point of view of its
caller.
3. Use a dedicated thread that sleeps until it is time to reconsider the
number of tasks, processes the queue of log messages, does any
adjustment, and then goes back to sleep. This is the most reliable in
terms of getting adjustments made when they are needed, but costs the
overhead of an extra thread that is just managing the thread pool.
Any views, alternatives?
Thanks,
Patricia
