frantisek holop wrote: > hmm, on Mon, Sep 14, 2009 at 10:23:58PM +0200, Claudio Jeker said that >> like to prove. In the end many of fefe's test programs did not actually >> measure what he assumed they would. > > and he was open to get patches to remedy those problems.
<sarcasm>and always showed total non-bias.</sarcasm> however, if you look at the tree, OpenBSD developers DID look at his test programs and for the ones that pointed out real issues, made considerable improve the performance in those areas. Oh, also in a lot of areas he didn't test. However, the goal was to do it better (in the broad sense of the term), not to top out on anyone's benchmark. I'm sure he updated his page to show the new results, right? > general dislike of any benchmark in the world is also part of the > openbsd culture just like some qualities of misc@ (although it's been > quite quiet lately). I think dislike of benchmarks is a general attribute of most experienced people on the technical side of things. Love of and trust in benchmarks is a general attribute of managers who want to pretend they made a good decision rather than a wild-ass guess based on who has the cuter sales rep and/or buys the better lunch (or sells products missing on their resume). > if the numbers were better, the general sentiment would > be rather different i believe. naw. Once you learn to loath benchmarks and how people use them, you can't start to cite them without your tongue ripping itself out of your mouth and beating you senseless. Even when you find one that sorta looks good, you start thinking about all the edge (or straight down the middle) cases it misses. One of my favorite benchmarks: I was approached by a network consultant who told me he was going to need an emergency DHCP server for an office and asked if I could have it done by the next day. I told him I could have it done in 30 minutes. He couldn't believe me when I told him that, so I invited him to watch me. So, I walked him through an OpenBSD install, and twenty minutes after the target machine (a very modest 400MHz Celeron) was first powered on, it was ready to be plugged in and serve DHCP. Still doesn't mean much. :) See the flaws in my "benchmark"? Of course you do. So do I. (got cocky a couple weeks later, told someone I could do a basic OpenBSD install on one of these things in "about five minutes". The machine I picked turned out to have a bad hard disk and did a massive number of disk retries before it finished loading. Ended up taking about seven minutes..and DID boot successfully. While I didn't hit my target, the spectator was still quite impressed :) > linux is faster in many respects (just look at zaurus) so what? > i dont use openbsd for its speed, but on the other hand i dont > downplay the importance of measuring things up and comparing it > with the others once in a while. i am sure speed in the end is > of councern, otherwise the os woudln't be in C but, whatchamacallit, > python. of course speed is a concern...to use one of my infamous car analogies, though...you need enough power and performance to get by safely. You don't want to drive a kiddie go-cart on the interstate freeway. However, once you can go twice the speed you ever need to go, what's the point in building a louder, more expensive and uncomfortable vehicle? Sure, some people go for that kinda thing, but that's why there are more than one kind of car on the market. (speaking of hypocracy, I do my darnedest to avoid being seen in a dull vehicle, though "speed" is only one definition of "not dull" :) Speed matters. Almost as much as some things, and nowhere near as much as others. > some things can be measured actually quite easily: how much content > a web server serves (not that much without sendfile()), how do the > databases perform, etc, this is all benchmark in the end, and the > programs doing the benchmarking are actually the daemons themselves. > so there, everyone is benchmarking 24/7 :] What is also worth measuring is what the ultimate bottleneck is. It is entertaining to hear people discuss benchmarks and performance, and make decisions based on those criteria, then realize they are sitting behind a pipe so skinny that a ten year old version of OpenBSD on a 17 year old 486 could saturate the link many times over, with no prospects of upgrading past what a Pentium II could handle in the reasonable future. SOME people DO need to filter or host or otherwise use multi-gigabit links. However making bad decisions based on unlikely scaling concerns is silly. As I've told people many times, "If you outgrow this, replacing it will be among the LEAST of your problems, look forward to it". Practically speaking, the people who need the performance at the edge of what OpenBSD can deliver usually are too busy to argue benchmarks. Nick.
