On 11/30/2010 1:43 AM, Peter Firmstone wrote:
Patricia Shanahan wrote:
Tom Hobbs wrote:
Yes, you're right.
I knew about the non-atomicity of ++, my concern was a call to reset
creeping in between the two parts of that operation.
That is a very good point.
Leaving reset unsynchronized, even with volatile, would lead to
results that would not be possible with full synchronization. Suppose
thread A is going to do a reset, and thread B is going to do an
increment, and everybody agrees the count is currently 10.
....
Ah yes, correct, my mistake, easy to stuff up isn't it? ;)
In essence I agree, unfortunately we don't know when we need the
performance, because we can't test scalability. I've only got 4 threads!
I've only got 8 threads on my largest system.
I am very, very strongly opposed to attempting performance tuning
without measurement. I've seen it tried many times over several decades,
and it is always a disaster. I've compared e.g. estimates of where
bottlenecks will be in an operating system prepared by the OS developers
to actual measurements, and I've yet to see the developers get it right.
That includes my own efforts at guessing bottlenecks, before I learned
that it is futile.
Without measurement, you get either get focused effort in entirely the
wrong places or diffuse effort spread over the whole system. What is
really needed is focused effort on a few real bottlenecks that will go
way beyond any set of rules that could reasonably be applied throughout
the system.
I believe the solution is to establish a working relationship with users
of River who have larger systems. They have a vested interest in helping
us make it more scalable.
Here are some simple tips I've found useful:
Have you measured the effectiveness of these tips on real world
scalability? What sort of gain do you see?
There may also be opportunities in the area of data structure and
algorithm scalability. For example, TaskManager uses an ArrayList to
represent something that is basically a FIFO with some reordering. That
is a good idea if, and only if, the queue lengths are always very short
even on large systems. However, I have no idea whether TaskManager queue
lengths tend to increase on large systems or not.
1. If all mutators are atomic and don't depend on previous state, a
volatile reference or field may be sufficient, but now we have
concurrency utilities, why not use an atomic reference or field
instead? Then if we find we later need a method based on previous
state, it's easily proven correct.
Do we have the concurrency utilities? The java.util.concurrent packages
are all "Since 1.5", and some of their classes are "Since 1.6". We can
only use them if we are abandoning any chance of River running with a
1.4 rt.jar.
To my mind, the advances in java.util and its sub-packages are a really
strong motivation for getting to 1.5 or, better, 1.6 ASAP.
Currently, a quick grep indicates java.util.concurrent is only used in
./qa/src/com/sun/jini/qa/harness/HeartOfTheMachine.java, which is part
of the QA infrastructure, not the production system.
...
Have you got any examples of a formal proof of correctness? Just out of
curiosity?
Unfortunately, the proofs of correctness I've written that related to
concurrency were based on confidential data and written on the job when
I was working as a performance architect for Cray Research and Sun
Microsystems.
I do have some coursework proofs from the UCSD Design of Algorithms
graduate course. I'll dig out an example to send to you directly.
Proof of correctness of large systems is a research topic. I'm talking
about looking very narrowly at a class that has been shown to be
performance-critical and is being subjected to special performance
tuning that uses risky techniques such as volatile.
Patricia
