also two revs of he BM have happened since that article..
rich
ken mays wrote:
The benchmark cannot be called a benchmark, as it does
not mention
what software has been compiled in which way (e.g.
which optimizer options
have been in effect).
For this reason, all performance differences mey be a
result of applying
different optimization when creating the test
binaries.
Jörg
Hi Jörg,
The performance differences might be 100% due to the
version of the compiler and optimizations used. My point
would be in what way is this significant to a user who sees
their audio files take 50% longer to encode on one product
versus the other? Do you think the average user will care
about those details or just the net result?
Regards,
Greg
Ref: http://www.phoronix.com/scan.php?page=article&item=os_threeway_2008&num=8
You are right in that the common 'user' just sees the end (i.e. 'net') result -
not the bottom-feeding details. Based on the Nov 24 benchmark
results I read last year, let us think about this article a bit more.
I'll point this out: Are the OS 2008.11 benchmarked performance results 'fixable' if a someone was to use the RIGHT compiler with the RIGHT optimizations to the challenge the benchmarks? That is my 'magic cookie' question.
The previous benchmark test of Nov 24, 2008 seemed to mention:
"If simply counting which operating system was in first place most frequently, it would be Ubuntu. Ubuntu 8.10 x86_64 was in first place eight times, OpenSolaris 2008.11 RC2 was in first place seven times, and FreeBSD 7.1 Beta 2 AMD64 was in first just three tests." - Ubuntu vs. OpenSolaris vs. FreeBSD Benchmarks, Published on November 24, 2008, Written by Michael Larabel.
So what happened here?!? I see these Nov 24th benchmarks tell a different story
than the newer February 09, 2009 benchmarks. If we look at FreeBSD 7.1 (final)
and OS 2008.11-b107 on today's terms and rerun these tests for ourselves - we'd
hope to see OS-2008.11-b107 in those same favorable results? Right? Now, if we
take Mac OS 10.5.6 and OS 2008.11-b107 and run these same benchmarks - would we
still think the OS 2008.11-b107 benchmarks were all faulty by optimizations?
Challenge Phoronix to recompare the benchmarks using Mac OS X 10.5.6 and OS
2008.11-b107, and Ubuntu 8.10_64 on the same hardware. We can even toss in a
benchmark for the optimized application binaries for OS 2008.11. Let us see if
the 'optimization theory' really was the problem versus the 'out-of-the-box'
debate... :o)
~ Ken Mays
_______________________________________________
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
_______________________________________________
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
