Come come now, lets not speak poorly of TPE, that is only one form of
expected behaviour, it happens to be very flexible, it's also only one
implementation of an ExecutorService.
For example if you use TPE with a BlockingQueue and a ThreadFactory (to
create daemon threads), where the blocking queue is of a fixed size, it
can be tuned to the environment it runs on.
TPE runs with the core pool size while the blocking queue isn't full,
however once it is, TPE adds more threads until it isn't, so the
blocking queue doesn't remain full, instead TPE finds a balance, where
you have a fixed size queue and sufficient threads that reach some point
of equilibrium between the core pool size and max threads.
You might set max threads based on the time you expect some of your
threads to be stalled and the number of available cpu threads. This you
can determine at runtime.
I've just committed a RunnableFuture that can be submitted to an
Executor, called DependantTask, that asks a PriorityHandler what tasks
must be executed first. The DependantTask and all the tasks it depends
upon can be executed asymmetrically by the queue, but when the
DependantTask is run, it submits itself to the PriorityHandler (an
interface) asking for dependent RunnableFuture's and runs them first,
many of them, may have already run from concurrent threads, however it
ensures that all dependent tasks complete first, by calling the Runnable
future get() method that waits for completion before runnning itself.
This is independent of the Executor implementation, as the Executor need
not concern itself with the dependencies, neither should the
DependantTask, that's the job of the PriorityHandler, which can work
across multiple Executors too.
Keep an open mind lads.
My +1 goes to Patrick, pickup the benefits of performance as platforms
improve. They've got more time & money than we'll ever have.
Cheers,
Peter.
Christopher Dolan wrote:
Wow, I did not know that about TPE. I need to go fix some code...
I withdraw my advocacy for switching TaskManager to use TPE.
Chris
-----Original Message-----
From: Gregg Wonderly [mailto:[email protected]]
Sent: Thursday, June 10, 2010 1:07 AM
To: [email protected]
Subject: Re: com.sun.jini.thread lock contention
As was kind of discussed earlier, TPE uses max threads in a different
way then
most people would think.
It creates up to min threads, as long as the "queue is not full"
(offer()
succeeding is "not full"). When the queue is "full", it creates up to
maxPoolSize threads for (maxPoolSize-corePoolSize) more adds.
So for an open ended queue, maxPoolSize has no meaning, and only
corePoolSize
threads will ever be created it looks to me.
I think there are still bugs to be fixed and behavior to be better
defined in
TPE in particular. I'm all for using the interfaces in j.u.c that make
sense to
use. But I am a bit worried about swapping out TaskManager for TPE
without a
lot more studying around the exact behaviors and failure modes, which
might help
identify some issues we need to address overall regarding how threads
are managed.
For example, the default life of 15mins on threads created by
TaskManager in its
current version, seems extreme. 15 seconds would be much better for
resource
management I think.
Gregg Wonderly
Patrick Wright wrote:
One point I'd like to raise about using java.util.concurrent and TPE:
I think that over the long term, it makes sense to (re)use existing
utilities which are being maintained by domain experts rather than
custom utilities you've written yourself. The concurrent libraries
available since Java 5 were written and maintained by people widely
recognized to be very, very good at a very hard problem. That doesn't
mean they, or the library, is perfect, just that there is value in
building on their work and letting them take care of the bugs and
optimizations over time. The downside would be that if a River user
was stuck with, say, Java 5, they couldn't take advantage of bugfixes
or improvements in Java 6. On the other hand, that's true of the
entire JDK.
The max threads issue seems to me a non-issue. A JVM can allocate only
so many native threads before it runs out of OS resources; that's a
hard limit. You can set a max of Integer.MAX_VALUE but your VM would
die long, long before it reached that.
For me this is more of design policy decision. Re-use, intelligently
and selectively, where possible, to reduce your project's workload.
Patrick
