Hi: Small update, I just created http://www.squeaksource.com/SMark.html
SMark is the attempt to build a common benchmarking framework for Smalltalk. It inspired from the metaphor used for unit testing in SUnit, and thus, a benchmark is implemented by adding #benchMyBenchmark to a subclass of SBenchmarkSuite. The code is originally based on PBenchmark the benchmark framework used for the PinocchioVM and RoarBenchmark a framework used for performance regression testing of the RoarVM. Other sources of inspiration are for instance the Caliper microbenchmarking framework for Java. (http://code.google.com/p/caliper/). The name choice of SMark is deliberately confusable with smark (as defined by the urban dictionary: Noun. A person who is being scamed but is in on the prank. Someone who knows they are being tricked. Its origin is from the term "mark" and is the shortened form of the phrase "smart mark".) freely following the old wisdom: "Lies, Damn Lies, and --Statistics-- Benchmarks" Current status is still pretty early, I am spec'ing out the behavior and data structures I imaging with tests. And I haven't actually renamed anything to new names yet. The next update here will be once there is something actually usable. However, the project is world-writeable, so feel free to have a look. Best regards Stefan On 13 Mar 2011, at 23:30, Stefan Marr wrote: > Hi: > > For the others, I am currently adapting the benchmarking infrastructure used > for Pinocchio to be a bit more general, and enable me to integrate our RoarVM > benchmarking tools. > The goal is to have a framework that allows all kind of benchmarking, written > like unit tests. > One other idea Henrik was interested in is to be able to easily compare the > benchmark results of different versions of method-implementations, to see > whether optimizations were successful. > > > Camillo, even so I don't have a working version yet, I was trying to commit > my refactoring (perhaps for review). > However, the PinocchioVM project seems to be 'global readonly'. > > Would it be better to make it a stand-alone project? > When we go with that step, as already mentioned, I would like to rename it. > > PBenchmark is a name that might prevent adoption. Not that I would like to go > into politics here, but perhaps we can consider a new name. > > Since the general idea is to write benchmarks like unit tests, how about > 'SMark'/'SBench' instead of 'SUnit'? > > Best regards > Stefan > > > PS: > > One thing, we might want to use as a source of inspiration in the future is: > http://code.google.com/p/caliper/ > > Thats a microbenchmark framework for Java, also using the unit-testing > metaphor. > There @Param is also neat > (http://code.google.com/p/caliper/source/browse/trunk/examples/src/main/java/examples/ArraySortBenchmark.java) > and allows to encode input sizes for which the benchmarks should be executed. > That is something I am not to interested in, but in case the framework finds > adoption that might be something to keep in mind. > > -- > Stefan Marr > Software Languages Lab > Vrije Universiteit Brussel > Pleinlaan 2 / B-1050 Brussels / Belgium > http://soft.vub.ac.be/~smarr > Phone: +32 2 629 2974 > Fax: +32 2 629 3525 > > -- Stefan Marr Software Languages Lab Vrije Universiteit Brussel Pleinlaan 2 / B-1050 Brussels / Belgium http://soft.vub.ac.be/~smarr Phone: +32 2 629 2974 Fax: +32 2 629 3525
