> > It'll run the experiment 17 times (17 for no particular reason) and > > return the median runtime. -n is the size and -c the cutoff.
> Median? Shouldn't you take the minimum? Are there any good papers > on benchmarking? I'm using cpucycles from: http://www.ecrypt.eu.org/ebats/cpucycles.html which advises: ------------------------ Notes on accuracy Benchmarking tools are encouraged to record several timings of a function: call cpucycles(), function(), cpucycles(), function(), etc., and then print one line reporting the differences between successive cpucycles() results. The median of several differences is much more stable than the average. Cycle counts continue to increase while other programs are running, while the operating system is handling an interruption such as a network packet, etc. This won't affect the median of several timings of a fast function---the function usually won't be interrupted---but it can affect the median of several timings of a slow function. Hopefully a benchmarking machine isn't running other programs. On dual-CPU systems (and dual-core systems such as the Athlon 64 X2), the CPUs often don't have synchronized cycle counters, so a process that switches CPUs can have its cycle counts jump forwards or backwards. I've never seen this affect the median of several timings. Some CPUs dynamically reduce CPU speed to save power, but deliberately keep their cycle counters running at full speed, the idea being that measuring time is more important than measuring cycles. Hopefully a benchmarking machine won't enter power-saving mode. Cycle counts are occasionally off by a multiple of 2^32 on some CPUs, as discussed below. I've never seen this affect the median of several timings. The estimate returned by cpucycles_persecond() may improve accuracy after cpucycles() has been called repeatedly. ------------------------ I just followed that. I guess if one takes the minimum or not depends on the data. If there are (much) easier cases then it is probably not best to take the minimum. However, I guess for matrix multiplication the times don't really depend on the entries of the matrices so I could use the minimum or just output all (minimum, average, median). > By the way -- present for you: > > http://m4ri.sagemath.org/ > > To update that page you have to ssh to sagemath.org. Cool, thanks! Any chance that I could get write access to the home directory so that I can upload my SSH public key? Martin -- name: Martin Albrecht _pgp: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x8EF0DC99 _www: http://www.informatik.uni-bremen.de/~malb _jab: [EMAIL PROTECTED] --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://www.sagemath.org -~----------~----~----~----~------~----~------~--~---
