I'm trying to track down abysmally low (15-30MB/s) write performance on a file system on a RAIDFframe level 1 RAID on mpt(4) SAS discs.
On a file system on one of the component devices, write performance is 140MB/s. On the RAID's raw device, I can write 110MB/s when using 64k blocks. When I INcrease the block size (like dd bs=128k), the write performance DROPs: 48MB/s at 128k, 20MB/s at 256k, 17 at 512, 15 at 1024. I can't make any sense out of this. Running iostat -x during the tests shows 64k (i.e. MAXPHYS) writes on the RAID, 8k (16 SpSU) on the components, no reads, all as expected. CPUs are close to idle. On a similar RAIDframe setup, but on SATA discs at on-board controllers, write throughput exceeds 100MB/s. All file systems are 16k fsbsize FFSv2 (i.e newfs -O 2 -b 16k) mounted -o log. What SAS controllers do other people use? Is mpii(4) ready for production? What about SAS hardware RAID controllers in JBOD mode?