Re: Performance Tracker project update

2008-02-13 Thread Antony Mawer
Erik Cederstrand wrote: Brooks Davis skrev: On Fri, Feb 08, 2008 at 09:41:09AM +0100, Erik Cederstrand wrote: I finally got around to testing this, and with a combination of mtree comparing md5 hashes, bsdiff compacting changed files and hardlinking unchanged files I get a reduction in size fr

Re: Performance Tracker project update

2008-02-13 Thread Erik Cederstrand
Dieter skrev: As suggested in other posts, deleting .pyo and .pyc files gets me down to 6MB. Static libraries (.a files) in /usr/lib and /usr/local/lib still have mismatching MD5 sums even though no source code change warrants this. Can I do anything about that? Perhaps they have an embedded ti

Re: Performance Tracker project update

2008-02-13 Thread Dieter
> As suggested in other posts, deleting .pyo and .pyc files gets me down to > 6MB. Static libraries (.a files) in /usr/lib and /usr/local/lib still have > mismatching MD5 sums even though no source code change warrants this. Can > I do anything about that? Perhaps they have an embedded timestamp o

Re: Performance Tracker project update

2008-02-13 Thread Brooks Davis
On Wed, Feb 13, 2008 at 03:46:10PM +0100, Erik Cederstrand wrote: > Brooks Davis skrev: >> On Fri, Feb 08, 2008 at 09:41:09AM +0100, Erik Cederstrand wrote: >>> I finally got around to testing this, and with a combination of mtree >>> comparing md5 hashes, bsdiff compacting changed files and hardl

Re: Performance Tracker project update

2008-02-13 Thread Erik Cederstrand
Brooks Davis skrev: On Fri, Feb 08, 2008 at 09:41:09AM +0100, Erik Cederstrand wrote: I finally got around to testing this, and with a combination of mtree comparing md5 hashes, bsdiff compacting changed files and hardlinking unchanged files I get a reduction in size from 256MB to 10MB. Pretty

Re: Performance Tracker project update

2008-02-08 Thread Ivan Voras
Chuck Swiger wrote: On Feb 8, 2008, at 12:43 PM, Ivan Voras wrote: Historically, the Python optimizer wasn't capable of doing much, true, but the more recent versions of the optimizer can actually do some peephole optimizations like algorithmic simplification and constant folding: http://docs.

Re: Performance Tracker project update

2008-02-08 Thread Chuck Swiger
On Feb 8, 2008, at 12:43 PM, Ivan Voras wrote: Historically, the Python optimizer wasn't capable of doing much, true, but the more recent versions of the optimizer can actually do some peephole optimizations like algorithmic simplification and constant folding: http://docs.python.org/whatsn

Re: Performance Tracker project update

2008-02-08 Thread Ivan Voras
Chuck Swiger wrote: Historically, the Python optimizer wasn't capable of doing much, true, but the more recent versions of the optimizer can actually do some peephole optimizations like algorithmic simplification and constant folding: http://docs.python.org/whatsnew/other-lang.html#SECTION00

Re: Performance Tracker project update

2008-02-08 Thread Chuck Swiger
On Feb 8, 2008, at 11:42 AM, Ivan Voras wrote: I'm not a python guru by any means, but I think .pyc files probably have data about the .py they are generated from because there's some sort of auto-generation available. It may be possible to not store them at all and just generate them before

Re: Performance Tracker project update

2008-02-08 Thread Ivan Voras
Brooks Davis wrote: On Fri, Feb 08, 2008 at 09:41:09AM +0100, Erik Cederstrand wrote: I finally got around to testing this, and with a combination of mtree comparing md5 hashes, bsdiff compacting changed files and hardlinking unchanged files I get a reduction in size from 256MB to 10MB. Pretty

Re: Performance Tracker project update

2008-02-08 Thread Brooks Davis
On Fri, Feb 08, 2008 at 09:41:09AM +0100, Erik Cederstrand wrote: > Brooks Davis skrev: >> On Wed, Jan 30, 2008 at 07:20:23PM +0100, Erik Cederstrand wrote: >>> >>> I'd like a situation where I can very quickly set up a slave with a >>> specific version of FreeBSD to run additional tests or provi

Re: Performance Tracker project update

2008-02-08 Thread Erik Cederstrand
Brooks Davis skrev: On Wed, Jan 30, 2008 at 07:20:23PM +0100, Erik Cederstrand wrote: I'd like a situation where I can very quickly set up a slave with a specific version of FreeBSD to run additional tests or provide shell access to a developer. This currently involves adding an entry to a qu

Re: Performance Tracker project update

2008-01-31 Thread gnn
At Wed, 23 Jan 2008 14:49:14 +0100, Erik Cederstrand wrote: > > Kris Kennaway wrote: > > > > This is coming along very nicely indeed! > > > > One suggestion I have is that as more metrics are added it becomes > > important for an "at a glance" overview of changes so we can monitor for > > perf

Re: Performance Tracker project update

2008-01-30 Thread Brooks Davis
On Wed, Jan 30, 2008 at 07:20:23PM +0100, Erik Cederstrand wrote: > Kris Kennaway wrote: >> Robert Watson wrote: >> One thing I am looking at is how to best create a library of world >> tarballs that can be used to populate a nfsroot (or hybrid of periodic >> tarballs + binary diffs to save space

Re: Performance Tracker project update

2008-01-30 Thread Erik Cederstrand
Kris Kennaway wrote: Robert Watson wrote: One thing I am looking at is how to best create a library of world tarballs that can be used to populate a nfsroot (or hybrid of periodic tarballs + binary diffs to save space). Then you could provide your benchmark in a standardized format (start/en

Re: Performance Tracker project update

2008-01-26 Thread Erik Cederstrand
Erik Cederstrand wrote: I'd like to send a small update on my progress on the Performance Tracker project. I now have a small setup of a server and a slave chugging along, currently collecting data. I'm following CURRENT and collecting results from super-smack and unixbench. The project s

Re: Performance Tracker project update

2008-01-24 Thread Erik Cederstrand
Alexander Leidinger wrote: Quoting Kris Kennaway <[EMAIL PROTECTED]> (from Thu, 24 Jan 2008 12:08:07 >> I don't see machine availability being a problem once we are ready to "take this live". Me too, but it's in the development stage and Robert asks for some new features, and I've read the a

Re: Performance Tracker project update

2008-01-24 Thread Alexander Leidinger
Quoting Kris Kennaway <[EMAIL PROTECTED]> (from Thu, 24 Jan 2008 12:08:07 +0100): Erik Cederstrand wrote: Alexander Leidinger wrote: In case you have some space left for more machines, maybe someone is willing to help out by sponsoring some. Just tell us how many machines you can hand

Re: Performance Tracker project update

2008-01-24 Thread Kris Kennaway
Ivan Voras wrote: Kris Kennaway wrote: Erik Cederstrand wrote: Ivan Voras wrote: I have a suggestion to make the graphs more readable: if a long period was chosen by the user (e.g. > 100 days / plot points), don't plot points and error bars, plot a simple line through the points. Also, set all

Re: Performance Tracker project update

2008-01-24 Thread Ivan Voras
Kris Kennaway wrote: > Erik Cederstrand wrote: >> Ivan Voras wrote: >>> >>> I have a suggestion to make the graphs more readable: if a long >>> period was chosen by the user (e.g. > 100 days / plot points), don't >>> plot points and error bars, plot a simple line through the points. >>> Also, set a

Re: Performance Tracker project update

2008-01-24 Thread Ivan Voras
Erik Cederstrand wrote: > I haven't touched malloc.conf but realize that I should. What's the > official recommendation on malloc settings? You'd have to patch /usr/src/lib/libc/stdlib/malloc.c and define MALLOC_PRODUCTION. Yes, it's not elegant. signature.asc Description: OpenPGP digital sig

Re: Performance Tracker project update

2008-01-24 Thread Kris Kennaway
Erik Cederstrand wrote: Alexander Leidinger wrote: In case you have some space left for more machines, maybe someone is willing to help out by sponsoring some. Just tell us how many machines you can handle (space/power/...) and if you are interested that we put it up on our wantlist. I'm s

Re: Performance Tracker project update

2008-01-24 Thread Kris Kennaway
Erik Cederstrand wrote: Ivan Voras wrote: I have a suggestion to make the graphs more readable: if a long period was chosen by the user (e.g. > 100 days / plot points), don't plot points and error bars, plot a simple line through the points. Also, set all date strings on the X-axis to empty

Re: Performance Tracker project update

2008-01-24 Thread Erik Cederstrand
Ivan Voras wrote: I have a suggestion to make the graphs more readable: if a long period was chosen by the user (e.g. > 100 days / plot points), don't plot points and error bars, plot a simple line through the points. Also, set all date strings on the X-axis to empty strings except for the da

Re: Performance Tracker project update

2008-01-24 Thread Erik Cederstrand
Alexander Leidinger wrote: In case you have some space left for more machines, maybe someone is willing to help out by sponsoring some. Just tell us how many machines you can handle (space/power/...) and if you are interested that we put it up on our wantlist. I'm sharing an office with 4 o

Re: Performance Tracker project update

2008-01-24 Thread Borja Marcos
On Jan 23, 2008, at 12:44 PM, Kris Kennaway wrote: One suggestion I have is that as more metrics are added it becomes important for an "at a glance" overview of changes so we can monitor for performance improvements and regressions among many workloads. One

Re: Performance Tracker project update

2008-01-24 Thread Alexander Leidinger
Quoting Erik Cederstrand <[EMAIL PROTECTED]> (from Wed, 23 Jan 2008 21:59:42 +0100): Finally, in the interests of making your life more complicated, it would be neat to graph performance across a set of FreeBSD branches overlaid or vertically offset so you could monitor, say, MySQL per

Re: Performance Tracker project update

2008-01-23 Thread Kris Kennaway
Kris Kennaway wrote: P.S. If I understand correctly, the float test shows a regression? The metric is calculations/second, so higher = better? The documentation on Unixbench is scarce, but I would think so. Interesting. Some candidate changes from 2007-10-02: Modified files: contrib

Re: Performance Tracker project update

2008-01-23 Thread Ivan Voras
Kris Kennaway wrote: The project still needs some work, but there's a temporary web interface to the data here: http://littlebit.dk:5000/plot/. Apart from the plotting it's possible to compare two dates and see the files that have changed. Error bars are 3*standard deviation, for the points wi

Re: Performance Tracker project update

2008-01-23 Thread Erik Cederstrand
Kris Kennaway wrote: Robert Watson wrote: Yes -- I was mostly thinking about backdating in order to play "catchup" when a new benchmark is introduced. One thing I am looking at is how to best create a library of world tarballs that can be used to populate a nfsroot (or hybrid of periodic t

Re: Performance Tracker project update

2008-01-23 Thread Erik Cederstrand
Robert Watson wrote: On Wed, 23 Jan 2008, Erik Cederstrand wrote: I agree that there's a need for an overview and some sort of notification. I've been collecting historical data to get a baseline for the statistics and I'll try to see what I can do over the next weeks. A thumbnail page of gr

Re: Performance Tracker project update

2008-01-23 Thread Kris Kennaway
Robert Watson wrote: I think it's best if participating machines supply data regularly for an extended period of time. Single or infrequent data points for a specific configuration don't make much sense. We need to compare apples to apples. Yes -- I was mostly thinking about backdating in or

Re: Performance Tracker project update

2008-01-23 Thread Robert Watson
On Wed, 23 Jan 2008, Erik Cederstrand wrote: Robert Watson wrote: This looks really exciting! Do you plan to add a way so that people can submit performance data? I.e., if I set up my own test box and want to submit a result once a week for that, will there be a way for me to get set up wit

Re: Performance Tracker project update

2008-01-23 Thread Erik Cederstrand
Robert Watson wrote: This looks really exciting! Do you plan to add a way so that people can submit performance data? I.e., if I set up my own test box and want to submit a result once a week for that, will there be a way for me to get set up with a username/password, submit configuration i

Re: Performance Tracker project update

2008-01-23 Thread Brooks Davis
On Wed, Jan 23, 2008 at 05:48:23AM +0100, Erik Cederstrand wrote: > Hi > > I'd like to send a small update on my progress on the Performance Tracker > project. > > I now have a small setup of a server and a slave chugging along, currently > collecting data. I'm following CURRENT and collecting

Re: Performance Tracker project update

2008-01-23 Thread Robert Watson
On Wed, 23 Jan 2008, Erik Cederstrand wrote: One way to do this would be a matrix of each metric with its change compared to recent samples. e.g. you could do a student's T comparison of today's numbers with those from yesterday, or from a week ago, and colour-code those that show a significa

Re: Performance Tracker project update

2008-01-23 Thread Robert Watson
On Wed, 23 Jan 2008, Erik Cederstrand wrote: I'd like to send a small update on my progress on the Performance Tracker project. I now have a small setup of a server and a slave chugging along, currently collecting data. I'm following CURRENT and collecting results from super-smack and unixb

Re: Performance Tracker project update

2008-01-23 Thread Kris Kennaway
Erik Cederstrand wrote: Kris Kennaway wrote: This is coming along very nicely indeed! One suggestion I have is that as more metrics are added it becomes important for an "at a glance" overview of changes so we can monitor for performance improvements and regressions among many workloads. >

Re: Performance Tracker project update

2008-01-23 Thread Erik Cederstrand
Kris Kennaway wrote: This is coming along very nicely indeed! One suggestion I have is that as more metrics are added it becomes important for an "at a glance" overview of changes so we can monitor for performance improvements and regressions among many workloads. > One way to do this would

Re: Performance Tracker project update

2008-01-23 Thread Kris Kennaway
Erik Cederstrand wrote: Hi I'd like to send a small update on my progress on the Performance Tracker project. I now have a small setup of a server and a slave chugging along, currently collecting data. I'm following CURRENT and collecting results from super-smack and unixbench. The projec