> > The non-profit Standard Performance Evaluation Corporation and > volunteers > > who work for it (including me) would be pretty dismayed that anyone > > believes that it has no relevance to performance evaluation. > > SPEC self-limits its relevance by refusing to recognize that it should > be open-source. being open-hostile means that it has very limited > numbers > of data points,
Yup, only 9,335 submissions indexed on this page: http://www.spec.org/cpu2006/results/cpu2006.html I'm not getting into a debate. I'm glad there are open source benchmarks available too. I'll just provide some facts and let readers decide for themselves. > very minimalistic UI (let alone data mining tools), It is a limited text-based UI -- that runs on Linux, Windows, and proprietary Unixes. Portability was/is a major goal of SPEC. The search form: http://www.spec.org/cgi-bin/osgresults?conf=cpu2006&op=form is useful for data mining. > and perhaps most importantly, slow adaptation to changes in how > machines > are used (memory footprint, etc). True. But SPEC MPI2007 v2.0 is a 2.4 GB package of software [larger datasets, and a 128 GB minimum RAM / 64 cores (min) requirement to run the Large suite; the Medium suite (16 GB, ~8 core minimum) is still part of 2.0]. I just downloaded Phoronix, and the tarball was ~450KB. It looks like a good collection of tests. Like HPC Challenge, and NAS Parallel, it does not provide a single number as a metric of performance. There are always compromises and knashing of teeth in coming up with a formula for that single number, but SPEC and the Linpack/Top500 maintainers have found that people like it. -Tom _______________________________________________ Beowulf mailing list, [email protected] sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
