2006/11/27, Geir Magnusson Jr. <[EMAIL PROTECTED]>:
Mikhail Loenko wrote:
>
> Not sure that when we elect an official benchmark we should take
> into account whether it's open or not. IMHO we should base our
> decision on how close it to the real life and how many people are
> interested in it
I agree for the latter - people are (for reasons I never grok)
interested in SPEC benchmark results, so I think we definitely need to
keep the eye on the ball for that one.
You will have a lot of trouble convincing me that SPEC benchmarks are
close to "real life", because if we're seeing SPECjbb results of around
100% of Sun SE 5, and then Dacapo showing us 40% of 'best', there seems
to be a problem.
In fact Dacapo and SPECjbb exercises different parts of API/VM stack.
So the scores are mostly irrelevant to each other. For instance if we
choose some awt benchmark it may show even worse result comparing to
Sun. This does not necessarily mean that this benchmark is better then
Dacapo or SPECjbb. :)
SY, Alexey
