> Now before I get long winded with details I'm think of, let me state up > front that I would like to gather your > ideas on the types of tests that should be done. >
Hi, I'm new to this list but have been thinking about benchmarking a little so hope I can contribute to the discussion. Couchdb already includes a small number of simple benchmarks in test/bench. I have submitted a pull request [1] to fix the harness so that running ./test/bench/run works. Adding some benchmarks for e.g. view build performance would be simple and would facilitate easy benchmarking per git commit. I also have an unpolished patch which will submit the results to a public couch. A simple web app could then roll the data into a nice chart to better understand performance changes by commit / by release / by os / by platform. > Profiles in this case refer to how the database is to be used, are we > reading alot, writng alot, logging, > highly concurrent, are we dealing with 5 concurrent session to 2000? Should > we do test over 1 hour, a day or a week? Along the lines of profiles my second idea is inspired from how the cairo graphics project does benchmarking. Cairo will record the drawing operations from real applications like firefox (e.g. fill circle, draw gradient, etc...) into a log and then the developer can play back the operations as a benchmark. Since couchdb is HTTP we could record the requests done by an app (e.g. from running an applications test suite) into a log and build up a collection of real world load simulations. Hope some of that is interesting. Adam [1] https://issues.apache.org/jira/browse/COUCHDB-1432 ps - I posted this with the gmane interface so hope it formats ok.
