Erik Stian Tefre wrote: > It was completely idle. Changing vfs.read_max to 80 triples the > sequential read performance, see bonnie++ output below (run on the same > box, nothing changed except vfs.read_max).
It might be that 3ware is specially pessimized by FreeBSD chopping IO into 64K blocks. But vfs.read_max doesn't change that so it's maybe not it. > bonnie++ with vfs_max=80: > Version 1.93d ------Sequential Output------ --Sequential Input- > --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP > /sec %CP > xxxxxxxxxxxxxxxx 1G 285 83 88193 28 26858 6 690 89 218642 59 > 406.5 5 > Latency 44353us 557ms 542ms 90795us 209ms 361ms > Version 1.93d ------Sequential Create------ --------Random > Create-------- > xxxxxxxxxxxxxxxxxxx -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > /sec %CP > 16 8204 19 +++++ +++ 22163 36 9925 25 +++++ +++ > +++++ +++ > Latency 585ms 571us 3844us 257ms 24015us 768us Are these numbers typical for 3ware's controllers? I still think something's bad about your setup, see the following performance results on a 3-drive RAID5 on Dell PERC5: Version 1.93d ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP xxxxx.xxx.xx 4000M 298 99 92319 25 30729 13 440 88 121370 25 533.1 29 Latency 28140us 711ms 430ms 528ms 74013us 225ms Version 1.93d ------Sequential Create------ --------Random Create-------- xxxxx.xxx.xx -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 30 18782 85 50106 99 45405 99 16481 58 60185 99 49638 99 Latency 259ms 21091us 130us 27529us 50us 75us I consider this ok, since (for the simple case of READing) the 3-drive RAID5 array has the performance of a 2-drive striped array. Your CPU usage is quite high (59% on sequential block input, if I'm reading it correctly) - are you limited by your CPU?
signature.asc
Description: OpenPGP digital signature