> > Seriously, all disks configured WriteThrough (spindle and SSD disks
> > alike)
> > using the dedicated ZIL SSD device, very noticeably faster than
> > enabling the
> > WriteBack.
>
> What do you get with both SSD ZIL and WriteBack disks enabled?
>
> I mean if you have both why not use both? Then both async and sync IO
> benefits.
Interesting, but unfortunately false. Soon I'll post the results here. I
just need to package them in a way suitable to give the public, and stick it
on a website. But I'm fighting IT fires for now and haven't had the time
yet.
Roughly speaking, the following are approximately representative. Of course
it varies based on tweaks of the benchmark and stuff like that.
Stripe 3 mirrors write through: 450-780 IOPS
Stripe 3 mirrors write back: 1030-2130 IOPS
Stripe 3 mirrors write back + SSD ZIL: 1220-2480 IOPS
Stripe 3 mirrors write through + SSD ZIL: 1840-2490 IOPS
Overall, I would say WriteBack is 2-3 times faster than naked disks. SSD
ZIL is 3-4 times faster than naked disk. And for some reason, having the
WriteBack enabled while you have SSD ZIL actually hurts performance by
approx 10%. You're better off to use the SSD ZIL with disks in Write
Through mode.
That result is surprising to me. But I have a theory to explain it. When
you have WriteBack enabled, the OS issues a small write, and the HBA
immediately returns to the OS: "Yes, it's on nonvolatile storage." So the
OS quickly gives it another, and another, until the HBA write cache is full.
Now the HBA faces the task of writing all those tiny writes to disk, and the
HBA must simply follow orders, writing a tiny chunk to the sector it said it
would write, and so on. The HBA cannot effectively consolidate the small
writes into a larger sequential block write. But if you have the WriteBack
disabled, and you have a SSD for ZIL, then ZFS can log the tiny operation on
SSD, and immediately return to the process: "Yes, it's on nonvolatile
storage." So the application can issue another, and another, and another.
ZFS is smart enough to aggregate all these tiny write operations into a
single larger sequential write before sending it to the spindle disks.
Long story short, the evidence suggests if you have SSD ZIL, you're better
off without WriteBack on the HBA. And I conjecture the reasoning behind it
is because ZFS can write buffer better than the HBA can.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
