On 4/21/10 2:15 AM, Robert Milkowski wrote: > I haven't heard from you in a while! Good to see you here again :) > > Sorry for stating obvious but at the end of a day it depends on what > your goals are. > Are you interested in micro-benchmarks and comparison to other file > systems? > > I think the most relevant filesystem benchmarks for users is when you > benchmark a specific application and present results from an > application point of view. For example, given a workload for Oracle, > MySQL, LDAP, ... how quickly it completes? How much benefit there is > by using SSDs? What about other filesystems? > > Micro-benchmarks are fine but very hard to be properly interpreted by > most users. > > Additionally most benchmarks are almost useless if they are not > compared to some other configuration with only a benchmarked component > changed. For example, knowing that some MySQL load completes in 1h on > ZFS is basically useless. But knowing that on the same HW with > Linux/ext3 and under the same load it completes in 2h would be > interesting to users. > > Other interesting thing would be to see an impact of different ZFS > setting on a benchmark results (aligned recordsize for database vs. > default, atime off vs. on, lzjb, gzip, ssd). Also comparison of > benchmark results with all default zfs setting compared to whatever > setting you did which gave you the best result.
Hey Robert... I'm always around. :) You've made an excellent case for benchmarking and where its useful.... but what I'm asking for on this thread is for folks to share the research they've done with as much specificity as possible for research purposes. :) Let me illustrate: To Darren's point on FileBench and vdbench... to date I've found these two to be the most useful. IOzone, while very popular, has always given me strange results which are inconsistent regardless of how large the block and data is. Given that the most important aspect of any benchmark is repeatability and sanity in results, I've found no value in IOzone any longer. vdbench has become my friend particularly in the area of physical disk profiling. Before tuning ZFS (or any filesystem) its important to find a solid baseline of performance on the underlying disk structure. So using a variety of vdbench profiles such as the following help you pinpoint exactly the edges of the performance envelope: sd=sd1,lun=/dev/rdsk/c0t1d0s0,threads=1 wd=wd1,sd=sd1,readpct=100,rhpct=0,seekpct=0 rd=run1,wd=wd1,iorate=max,elapsed=10,interval=1,forxfersize=(4k-4096k,d) With vdbench and the workload above I can get consistent, reliable results time after time and the results on other systems match. This is particularly key if your running a hardware RAID controller under ZFS. There isn't anything dd can do that vdbench can't do better. Using a workload like above both at differing xfer sizes and also at differing thread counts really helps give an accurate picture of the disk capabilities. Moving up into the filesystem. I've been looking intently at improving my FileBench profiles, based on the supplied ones with tweaking. I'm trying to get to a methodology that provides me with time-after-time repeatable results for real comparison between systems. I'm looking hard at vdbench file workloads, but they aren't yet nearly as sophisticated as FileBench. I am also looking at FIO (http://freshmeat.net/projects/fio/), which is FileBench-esce. At the end of the day, I agree entirely that application benchmarks are far more effective judges... but they are also more time consuming and less flexible than dedicated tools. The key is honing generic benchmarks to provide useful data which can be relied upon for making accurate estimates as regards to application performance. When you start judging filesystem performance based on something like MySQL there are simply too many variables involved. So, I appreciate the Benchmark 101, but I'm looking for anyone interested in sharing meat. Most of the existing ZFS benchmarks folks published are several years old now, and most were using IOzone. benr. _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
