Cool stuff, Robert. It'd be interesting to see some RAID-Z (single- and
double-parity) benchmarks as well, but understandably this takes time
;-)
The first thing to note is that the current Nevada bits have a number of
performance fixes not in S10u2, so there's going to be a natural bias
when comparing ZFS to ZFS between these systems.
Second, you may be able to get more performance from the ZFS filesystem
on the HW lun by tweaking the max pending # of reqeusts. One thing
we've found is that ZFS currently has a hardcoded limit of how many
outstanding requests to send to the underlying vdev (35). This works
well for most single devices, but large arrays can actually handle more,
and we end up leaving some performance on the floor. Currently the only
way to tweak this variable is through 'mdb -kw'. Try something like:
# mdb -kw
> ::spa -v
ADDR STATE NAME
ffffffff82ef4140 ACTIVE pool
ADDR STATE AUX DESCRIPTION
ffffffff9677a1c0 HEALTHY - root
ffffffff9678d080 HEALTHY - raidz
ffffffff9678db80 HEALTHY - /dev/dsk/c2d0s0
ffffffff96778640 HEALTHY - /dev/dsk/c3d0s0
ffffffff967780c0 HEALTHY - /dev/dsk/c4d0s0
ffffffff9e495780 HEALTHY - /dev/dsk/c5d0s0
> ffffffff9678db80::print -a vdev_t vdev_queue.vq_max_pending
ffffffff9678df00 vdev_queue.vq_max_pending = 0x23
> ffffffff9678df00/Z0t60
0xffffffff9678df00: 0x23 = 0x3c
This will change the max # of pending requests for the disk to 60,
instead of 35. We're trying to figure out how to tweak and/or
dynamically detect the best value here, so any more data would be
useful.
- Eric
On Mon, Aug 07, 2006 at 08:22:24AM -0700, Robert Milkowski wrote:
> Hi.
>
> 3510 with two HW controllers, configured on LUN in RAID-10 using 12
> disks in head unit (FC-AL 73GB 15K disks). Optimization set to random,
> stripe size 32KB. Connected to v440 using two links, however in tests
> only one link was used (no MPxIO).
>
> I used filebench and varmail test with default parameters and run for
> 60s, test was run twice.
>
> System is S10U2 with all available patches (all support patches), kernel -18.
>
>
> ZFS filesystem on HW lun with atime=off:
>
> IO Summary: 499078 ops 8248.0 ops/s, (1269/1269 r/w) 40.6mb/s, 314us
> cpu/op, 6.0ms latency
> IO Summary: 503112 ops 8320.2 ops/s, (1280/1280 r/w) 41.0mb/s, 296us
> cpu/op, 5.9ms latency
>
> Now the same LUN but ZFS was destroyed and UFS filesystem was created.
> UFS filesystem on HW lun with maxcontig=24 and noatime:
>
> IO Summary: 401671 ops 6638.2 ops/s, (1021/1021 r/w) 32.7mb/s, 404us
> cpu/op, 7.5ms latency
> IO Summary: 403194 ops 6664.5 ops/s, (1025/1025 r/w) 32.5mb/s, 406us
> cpu/op, 7.5ms latency
>
>
>
> Now another v440 server (the same config) with snv_44, connected several 3510
> JBODS on two FC-loops however only one loop was used (no MPxIO). The same
> disks (73GB FC-AL 15K).
>
> ZFS filesystem with atime=off with ZFS raid-10 using 12 disks from one
> enclosure:
>
> IO Summary: 558331 ops 9244.1 ops/s, (1422/1422 r/w) 45.2mb/s, 312us
> cpu/op, 5.2ms latency
> IO Summary: 537542 ops 8899.9 ops/s, (1369/1369 r/w) 43.5mb/s, 307us
> cpu/op, 5.4ms latency
--
Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock
_______________________________________________
zfs-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
