So all caching is disabled. It might even have HDD cache disabled. I am not surprised at these numbers. You should go to the bios and enable Write Back cache and adaptive readahead; you should also enable drive cache while at it. It that knob isn't in the BIOS update your firmware.
On Wed, Mar 04, 2009 at 06:30:47PM -0500, Denis Hainsworth wrote: > Thanks for your response. Sorry I left out any details in my intial > email. Also wanted to thank you Marco for all the work you've done for > openbsd. > > > We are running a fairly basic bonnie++ test like so > bonnie++ -d /path -s 12000 -m machinename -u 0:0 > > Here are the full details: > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec > %CP > machine1 12000M 6131 3 6470 1 6220 1 72017 51 103703 19 361.3 > 0 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec > %CP > 16 1423 93 +++++ +++ 8049 27 1434 93 1689 99 4005 > 98 > machine1,12000M,6131,3,6470,1,6220,1,72017,51,103703,19,361.3,0,16,1423,93,++++ > > +,+++,8049,27,1434,93,1689,99,4005,98 > > > This is on a 388GB partition that is made up out of 4disks in a RAID5 > config. Here are the mount options we have been using: > type ffs (local, noatime, softdep) .. we've tried turing off softdep, > but it did not help. This filesystem was initialized using 4k inodes. > Increasing the inode size also did not seem to help. > > Due to the fact that we have not gotten the megacli working, we > have not made any changes to the raid options. I can at least read the > options sorta using LSI's builtin Megacli in their newer firmware. It > shows that the cache policy is set to: > > WriteThrough, ReadAheadNone, Direct, NoWrite if Bad BBU (I confirmed the > BBU shows as good) > Write Policy: WT (writethrough I presume) > Read Policy : None > > We threw openbsd 4.4 on a spare machine we dug up. This machine is > phsically identical to the ones we are having issues with. > > We did not see the same write problems .. here is a similar bonnie run, > against a standard openbsd filesystem mounted as type ffs (local, > nodev) this files system was initialized using 8k inodes. > > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec > %CP > machine2 12000M 31174 13 40551 5 10003 1 60582 35 82658 7 381.1 0 > ------Sequential Create------ --------Random Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 850 1 +++++ +++ 1848 1 868 1 +++++ +++ 344 0 > machine2,12000M,31174,13,40551,5,10003,1,60582,35,82658,7,381.1,0,16,850,1,+++ > +++,+++,1848,1,868,1,+++++,+++,344,0 > > > Additionally as I stated before our previous version of hardware which is > nearly identical and running the same openbsd 4.2 setup also performs fine. > It has the same 4k inode filesystem mounted with (local, noatime, softdep) > options. > > Version 1.03 ------Sequential Output------ --Sequential Input- > --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- > --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec > %CP > machine3 12000M 23656 20 30849 20 10298 5 29102 37 36291 12 225.0 > 1 > ------Sequential Create------ --------Random > Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- > -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec > %CP > 16 1206 92 +++++ +++ 16083 45 1070 81 1417 97 2991 > 88 > machine3,12000M,23656,20,30849,20,10298,5,29102,37,36291,12,225.0,1,16,1206,92,+ > > ++++,+++,16083,45,1070,81,1417,97,2991,88 > > > > So since machine1 and machine3 are running the same exact software it would > seem to indicate a hardware issue. Yet the same hardware running openbsd > 4.4 also seems ok which tends to indicate some odd interaction between the > hardware and openbsd 4.2. > > So if anyone has any thoughts they would be appreciated as I have been > going around in circles on this issue for a while now. > > Lastly Marco mentioned I might have interrupt issues. How would I check > for that? > > -denis > > Os Tue, Mar 03, 2009 at 10:21:51PM -0600, Marco Peereboom wrote: > > I think I have to conclude that you (or your hardware) are doing it > > wrong. Either you have interrupt issues or your raid card setup wrong. > > > > I also have no idea how you measured these numbers so I can't deduce > > any validity from them. > > > > On Tue, Mar 03, 2009 at 05:45:02PM -0500, Denis Hainsworth wrote: > > > Hello, > > > > > > Some what of a novice openbsd user and hope I didnt miss something > > > obvious. I > > > have a box running openbsd 4.2 which I cannot upgrade for various > > > reasons. The machine has the Symbios Logic MegaRAID SAS 1064R > > > controller and currently has a 4 disk Raid5 set up in bios with Seagate > > > ST9146802SS disks. We also run the same machines with Linux and > > > openbsd 4.2 with older hardware using an Ultra Scsi controller. > > > > > > Anyway we have some basic tests we run and noticed that a basic Bonnie++ > > > test on all 3 machines gave the following performance numbers. > > > > > > Openbsd 4.2 and Symbios SAS > > > 6073 K/sec Block Seq Output and 83086 K/sec Block Seq Input > > > Linux RH 7 and Symbios SAS > > > 65096 K/sec Block Seq Output and 165194 K/sec Block Seq Input > > > Openbsd 4.2 and Adaptec SCSI > > > 30849 K/sec Block Seq Output and 36291 K/sec Block Seq Input > > > > > > Now I wouldnt expect the numbers to match exactly given a number > > > parameters but the 6073 K/sec Block Seq Output is somewhat of a > > > concern. Since it doesnt seem to affect Linux using the exact same > > > hardware I think its most likly a driver and or configuration issue. > > > So does anyone know if there are issues with the mfi driver or any > > > configuration bits within openbsd I might be missing with regaurds to > > > disk performance? > > > > > > Here's the exact line from syslog if thats of any help: > > > Feb 18 19:45:33 /bsd: mfi0 at pci4 dev 14 function 0 "Symbios Logic > > > MegaRAID SAS 1064R" rev 0x00: apic 8 int 18 (irq 5) > > > Feb 18 19:45:33 /bsd: mfi0: logical drives 1, version 5.1.1-0038, 128MB > > > RAM > > > > > > I will note that we needed to apply a driver patch to recognise this > > > card in 4.2 but that was simply to add the pci device type to the list > > > of other supported types. I am unaware of any other changes. > > > > > > thank you for your time > > > -denis > > > > > > -- > > > __________________________ > > > Denis Alan Hainsworth > > > de...@alumni.brandeis.edu > > -- > __________________________ > Denis Alan Hainsworth > de...@alumni.brandeis.edu
