On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
I'm having trouble squaring this with the description of
ThreadPoolExecutor. If I'm reading it right, it *can* be initialized to
set a low limit on the maximum number of threads, but can equally well
be set to have up to Integer.MAX_VALUE threads, effectively unlimited.
Yes, this is the case. There are some other subtle behaviors which we just need
to study to make sure that the "documented behavior" is what we need.
There are two forms of TaskManager behavior that I do not see an easy
way to reproduce using a simple ThreadPoolExecutor, runAfter and loadLevel.
runAfter my be better handled by the class issuing the tasks.
There is also the Fork-Join framework which does allow some "arrangement" of
order. I'm not sure this is a great fit right now, but it is something to
consider I think, especially for "notifications", "commits" and some other
things that have two level ordering "function->tasks".
loadLevel permits specification of a ratio between threads and tasks, so
that threads are only created up to e.g. one third of the number of
tasks. I'm not sure that is a really good idea. It means leaving tasks
to wait even if we could create another thread.
I don't like this thought at all in a distributed system. A vast majority of
tasks really need to be invoked immediately to maintain a reasonable chance at
throughput.
Gregg
Patricia
On 6/29/2011 7:06 AM, Christopher Dolan wrote:
Just remember, we discussed this a year ago
(http://mail-archives.apache.org/mod_mbox/incubator-river-dev/201006.mbo
x/%3c4c12147a.8030...@zeus.net.au%3E) and some of us concluded that
ThreadPoolExecutor has rather non-intuitive runtime behavior because
it's optimized to not cause OutOfMemory problems. Not everyone had a
problem with that behavior, but I'm not a fan.
Chris
P.S. can someone add a link to the Incubator mail archives to the web
page? They are quite hard to find...
http://mail-archives.apache.org/mod_mbox/incubator-river-dev/
http://mail-archives.apache.org/mod_mbox/incubator-river-user/
http://mail-archives.apache.org/mod_mbox/incubator-river-commits/
-----Original Message-----
From: Peter Firmstone [mailto:j...@zeus.net.au]
Sent: Sunday, June 26, 2011 5:18 PM
To: dev@river.apache.org
Subject: Re: TaskManager
I think the way to look at this is to examine the non-trivial runAfter
implementations. If they have common elements that can be helped by
some
utility classes, write those classes. If not, deal with each
individually.
We are now committed to 1.5, so if we did not have runAfter we could
replace TaskManager with java.util.concurrent.ThreadPoolExecutor.
That's probably where we should be headed, since it reduces our
maintenance burden.
Peter.
