Kris Kennaway wrote:
This is coming along very nicely indeed!
One suggestion I have is that as more metrics are added it becomes
important for an "at a glance" overview of changes so we can monitor for
performance improvements and regressions among many workloads.
>
One way to do this would be a matrix of each metric with its change
compared to recent samples. e.g. you could do a student's T comparison
of today's numbers with those from yesterday, or from a week ago, and
colour-code those that show a significant deviation from "no change".
This might be a bit noisy on short timescales, so you could aggregrate
data into larger bins and compare e.g. moving 1-week aggregates.
Fluctuations on short timescales won't stand out, but if there is a real
change then it will show up less than a week later.
I agree that there's a need for an overview and some sort of
notification. I've been collecting historical data to get a baseline for
the statistics and I'll try to see what I can do over the next weeks.
These significant events could also be graphed themselves and/or a
history log maintained (or automatically annotated on the individual
graphs) so historical changes can also be pinpointed.
At some point the ability to annotate the data will become important
(e.g. "We understand the cause of this, it was r1.123 of foo.c, which
was corrected in r1.124. The developer responsible has been shot.")
There's a field in the database for this sort of thing. I just think it
needs some sort of authentication. That'll have to wait a bit.
P.S. If I understand correctly, the float test shows a regression? The
metric is calculations/second, so higher = better?
The documentation on Unixbench is scarce, but I would think so.
BTW if anyone's interested my SVN repo is online at:
svn://littlebit.dk/website/trunk (Pylons project)
svn://littlebit.dk/tracker/trunk (sh/Python scripts for runnning the
server and slaves)
Be careful with your eyes - this is my first attempt at both shell
scripting and Python :-)
Erik
_______________________________________________
freebsd-performance@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-performance
To unsubscribe, send any mail to "[EMAIL PROTECTED]"