Robert Watson wrote:The next thing that would be quite nice to measure is the rate of I/O transactions per second we can get to the disk using the disk device directly, with a minimal transaction size. I have a vague recollection that you have to be careful in Linux because their character device nodes for disk devices are buffered, and you really want unbuffered I/O.
I noticed that changing vfs.read_max from the default 8 to 16 has a dramatic effect on sequential read performance. Increasing it further did not have measurable effect.
Wow. Setting vfs.read_max=16 resulted in a 50+% performance boost when sequentially reading from my 4-disk RAID5 (ida(4)) via the filesystem. Write performance was unchanged. Raw device read performance was also unchanged (~40 MB/s). With vfs.read_max>8, reading through the filesystem is *faster* than reading from the raw device!
Note that these tests were run in multi-user on an idle 5.4-RC4 machine with 1 GB of RAM and quad P3 Xeons. The machine is currently using ULE. These tests were repeated 3 times with results in the +-0.1MB/s range.
$ sysctl vfs.read_max
vfs.read_max: 8
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 185.571466 secs (35315774 bytes/sec)
3m5.58s real 0.16s user 1m0.89s sys
$ sysctl vfs.read_max=16
vfs.read_max: 8 -> 16
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 118.846665 secs (55143323 bytes/sec)
1m58.85s real 0.21s user 1m3.25s sys
$ sysctl vfs.read_max=24
vfs.read_max: 16 -> 24
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 107.360600 secs (61042878 bytes/sec)
1m47.36s real 0.14s user 1m3.27s sys
$ sysctl vfs.read_max=32
vfs.read_max: 24 -> 32
$ /usr/bin/time -h dd of=/dev/null if=/usr/testfile bs=64k count=100000
100000+0 records in
100000+0 records out
6553600000 bytes transferred in 102.288822 secs (64069562 bytes/sec)
1m42.29s real 0.20s user 1m4.52s sys-- Jonathan Noack | [EMAIL PROTECTED] | OpenPGP: 0x991D8195
signature.asc
Description: OpenPGP digital signature
