-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 09/29/2011 04:51 PM, John Cowan wrote: > Alaric Snell-Pym scripsit: > >> If the supposed performance improvement can't be benchmarked, then >> it's pointless, as nobody will actually benefit from it. Any case >> where somebody can benefit from a performance improvement can be >> turned into a benchmark that consists of running the code that is sped >> up, and timing it. >> >> Benchmarks are like unit tests; they are snippets of code that perform >> some operation but, rather than testing correct responses, their >> emphasis is on testing resource usage. > > Your clarification down-thread that a benchmark can be of any size makes > this comparison rather otiose. Nobody is going to have a benchmark > suite that includes tests like these: > > With patch #1234, application 'foo' runs in an acceptable 18 hours > rather than an intolerable 25 hours. (Obviously the improvement has to > be nonlinear.)
Actually, where I work we do! The full test suite takes all weekend - on a cluster of fairly beefy hardware, running different bits in parallel. But there's a hierarchy of tests and benchmarks. The correctness tests we run on our laptops before committing code "to the trunk" take fifteen minutes, and we run benchmarks in the five-minute range for quickly checking the results of changes. The full suite runs only once a week... ABS - -- Alaric Snell-Pym http://www.snell-pym.org.uk/alaric/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk6EmFQACgkQRgz/WHNxCGp2IwCfcofES+0R2BLmTBZ18wStt5Yf /zUAn3Z0NYBhWzWrDodOF6+gPi3441k5 =DVDu -----END PGP SIGNATURE----- _______________________________________________ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users