Robert Clipsham Wrote:

> Georg Wrede wrote:
> > Suppose you have run the same program very recently before the test. 
> > Then the executable will be in memory already, any other files it may 
> > want to access are in memory too.
> > 
> > This makes execution much faster than if it were the first time ever 
> > this program is run.
> > 
> > If things were deterministic, then you wouldn't run several times and 
> > average the results, right?
> 
> Ok, I will rerun the tests later today and disregard the first test. I 
> may also take the minimum value rather than taking an average (thanks to 
> Bill Baxter for this idea).


As you're re-inventing functionality that's in the benchmarks game measurement 
scripts, let me suggest that there are 2 phases involved:

1) record measurements
2) analyze measurements

As long as you keep the measurements in the order they were made in and keep 
the measurements for each different configuration in their own file, you can 
decide to do different selections from those measurements at some later date.

You can throw away the first measurement or not, you can take the fastest or 
the median, you can ... without doing new measurements.

As you are only trying to measure a couple of language implementations, measure 
them across a dozen different input values rather than one or two - leaving the 
computer churning overnight will help keep your home warm :-)

Reply via email to