(Resuming a few past discussions about std.benchmark.)
On 4/9/12 1:26 PM, Manfred Nowak wrote:
Andrei Alexandrescu wrote:
I disagree with running two benchmarks in parallel because that
exposes them to even more noise (scheduling, CPU count, current
machine load etc).
I did not mean to run two or more benchmarks in parallel but only
more instances of the benchmarked program _without_ the environment
supplied by the benchmakr. Of course may there be more noise. But if
so, that noise is dependent on the additional running instances and
to nothing else.
I doubt that's the case. Once we discuss running several instances in
parallel, I predict the vagaries of scheduling become much more prevalent.
Now let T(n) be the time needed to run n instances in parallel on a
single processor. According to your definition and your remark above:
§ T(n) = n*A + Q + N + P(n)
where P(n)>=0 and P(1)==0 is the additional noise with which the
benchmarked program disturbs itself.
Please observe that Q and N eliminate themselves on subtraction:
§ T(2) - T(1) = A + P(2) - P(1)
§ T(3) - T(2) = A + P(3) - P(2)
...
P(n+1)-P(n) measures the sensitivity of the benchmarked program to
its one noise. As you wrote yourself its value should be close to
zero until the machine is close to falling flat.
It would be great if that applied, but I'm afraid things get
significantly more complex. For example, the noise will have a
per-processor component, and the processes will compete for cache among
themselves.
Andrei
