On Mon, Jul 19, 2010 at 11:18 AM, Nadav Har'El <n...@math.technion.ac.il>wrote:
> On Sun, Jul 18, 2010, Elazar Leibovich wrote about "Re: New Freecell Solver > gcc-4.5.0 vs. LLVM+clang Benchmark": > > The standard deviation can give you an estimation of the minimal running > > time. (99.xxxx of the samples are within X standard deviations below the > > average. Pick a high enough xxxx relative to the number of times you'll > run > > the software, and you'll get an estimation of the minimum running time > > you'll get). > > > > The scalar representing the minimum running time have the obnoxious > > mis-feature that it doesn't tend to a certain value. So with a small > >... > > Usually, the run time of deterministic programs does not behave the way > you describe (i.e., with a gaussian distribution, having an average and > distributed around it). A deterministic program *DOES* have a clearly > defined minimum run time, which it will theoretically achieve on every > run, unless delayed by something else - other processes, daemons in the > background, or whatever. > > Imagine, for example, that you run a certain program 5 times and get the > times: 20.0, 18.0, 18.1, 27.0, 18.1 > Evidently, the first run was slower because things were not in the cache, > and the run that took 27.0 was delayed by some other process in the > background > taking up the CPU or disk. The minimum run time, 18.0, is the most > interesting > one - it is the time a run would take every time, if things were perfect. > If you average the above numbers, or find the standard deviation, etc., > the numbers would not be very interesting... > I take your point for a very simple program like the freecell solver which is mainly CPU-memory bound. Thanks for the input. However I believe that every complicated enough program might have a pretty big randomness factor. Consider for example the physical disk layout in the current run, which is affected by the long history of disk usage in the last year. Or the memory allocation which is affected by the current memory layout which also depends on the memory allocation history of this particular computer. I'm not talking about GC which is highly non-deterministic, but still need to be taken into account (if I'm considering a software with GC, I want to find its performance even when it runs, since the probability the GC will run depends on how well I wrote my code). Those are all random (or hard to predict) functions which can affect the running time of even a seemingly deterministic program. > > -- > Nadav Har'El | Monday, Jul 19 2010, 8 Av > 5770 > n...@math.technion.ac.il > |----------------------------------------- > Phone +972-523-790466, ICQ 13349191 |How long a minute depends on what side > of > http://nadav.harel.org.il |the bathroom door you're on. >
_______________________________________________ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
