Looks like a good list. I'd also suggest sticking to Erlang as well. You might want to check out basho_bench for one approach to load testing with Erlang. Even if you don't use it directly its a good example of how you might structure such a thing.
On Thu, Mar 22, 2012 at 7:58 PM, Jason Smith <[email protected]> wrote: > Hi, Mike. This is excellent! > > I hate to multiply the effort, but almost every bullet-point you list > could also be tested concurrently. In other words, concurrent > performance is not a line-item to test, it is a whole new category of > testing: view building with concurrent load, concurrent filtered > replication, concurrent updates, with erlang or JS validation > functions, etc. > > That is clearly harder to test, and it's fine to make it out of scope > for now; however, **those** results are really the most useful when > deciding whether to bet the farm. > > On Fri, Mar 23, 2012 at 7:54 AM, Mike Coolin <[email protected]> wrote: >> Hi all, >> >> I have been looking at couchdb for a couple of months now and really like >> what i see, in fact I've started developing my project using couchapp. >> However I do find some areas where I would like to better understand how >> well or badly couch performs. >> >> So I am proposing starting a project that will implement some testing tools >> in erlang to test several aspects of couch, in my case I am looking for >> numbers for both the web server and the database. >> >> Now before I get long winded with details I'm think of, let me state up >> front that I would like to gather your ideas on the types of tests that >> should be done. >> >> I firmly believe that many applications will have differing usage profiles, >> what I want to try to capture is a number of common profiles, now you may >> ask what do I mean by a profile? >> >> Profiles in this case refer to how the database is to be used, are we >> reading alot, writng alot, logging, highly concurrent, are we dealing with 5 >> concurrent session to 2000? Should we do test over 1 hour, a day or a week? >> >> Ideally this tool would provide a means of testing the various operating >> system settings/erLang settings assisting production admins in configuring >> thier environments to perform optimally for them. It could also assist >> developers in testing code and tuning it for optimal performance and safety. >> >> I see a number of areas that can be tested including: >> * Web server performance >> >> * File serving >> * Put/get/delete/post >> * load testing >> * Database performance >> >> * creating tables >> * populating tables >> * view performance - javascript - erlang >> * filter performance >> * insert with system keys and ideal keys >> * inserts with bulk inserts >> * database size >> * varying record sizes >> * concurrent user load testing >> * attachments processing >> * replication >> * database compaction >> * bulk inset/view response >> Ideally I'd like to record the erlang/system setup and whether the database >> and test code were running on the same system or separate systems and result >> sets. In this way others would be able to evaluate couchdb with more >> realistic expectations and couchdb will have something solid to point to in >> terms of its performance. >> >> I have not yet created the project on git, but that is my intension, but as >> I learned last night its nice to toss an idea out there, it's likely someone >> has some interesting things I could start with. >> >> Please add any ideas/issues/objections to this email. If you like the idea >> and think it would be useful let me know, if not, let me know as well. >> >> So far modeling something after nodeload was suggested, I briefly looked at >> it and yes it looks nice. But I'd like to take advantage of erlangs ability >> to run multiple processes to really load test the server. >> >> Looking forward to your comments. >> >> Thanks Mike > > > > -- > Iris Couch
