On Mon, Apr 14, 2008 at 10:07:14AM +0100, Alberto Simoes wrote:
> Bob Rogers wrote:
> > From: Alberto Simoes <[EMAIL PROTECTED]>
> > > 3. A semi-log plot would be easier to interpret.
> >
> > Smash tried a log plot, but it wasn't easier to interpret. I am not
> > sure what is a semi-log plot.
> >
> >Logarithmic on the Y axis, linear on the X. (There may be a better term
> >for this, but if so, I can't think of it right now.)
>
> Oh. That was what we done. It is easier to read values. It is more
> difficult to understand growing speed (derivative) just looking into it.
The derivative ("growing speed") of the curves in the bintree
benchmark says more about the complexity of the underlying
algorithm than the efficiency of the various implementations.
In the case of the bintree benchmark, a log plot would make it
easier to see what is really happening. The linear Y-axis makes
it appear as though things get progressively worse for Parrot
for larger tree heights, but the ratio of Parrot's performance
to the others seems to remain fairly constant -- i.e., Parrot
0.6.0 takes roughly twice as long as Perl 5 for this benchmark,
for any input size. Thus the relationship between the two should
look more linear than curved.
In fact, for all of the benchmarks that are based on increasing
number of runs it might be more useful to display the results as
performance ratio rather than report absolute times. For example,
we could (arbitrarily) pick Perl 5 as having a performance of 100
for any given input size, and if Parrot takes twice as long at that
same input its score would be 200, if Python takes 3/4ths as long
its score would be 75. This would allow the graphs to highlight
the efficiency of the implementations rather than the inherent
complexity of the underlying algorithms. Of course, if any of
the implementations started to show substantially improved/degraded
performance at some given input size, such a graph would show
that also.
Pm
