On Tue, 30 Sep 2008, Heikki Linnakangas wrote:

Doesn't some filesystems include a per-block CRC, which would achieve the same thing? ZFS?

Yes, there is a popular advoacy piece for ZFS with a high-level view of why and how they implement that at http://blogs.sun.com/bonwick/entry/zfs_end_to_end_data The guarantees are stronger than what you can get if you just put a CRC in the block itself. I'd never really thought too hard about putting this in the database knowing that ZFS is available for environments where this is a concern, but it certainly would be a nice addition.

The best analysis I've ever seen that makes a case for OS or higher level disk checksums of some sort, by looking at the myriad ways that disks and disk arrays fail in the real world, is in http://www.usenix.org/event/fast08/tech/full_papers/bairavasundaram/bairavasundaram.pdf (there is a shorter version that hits the high points of that at http://www.usenix.org/publications/login/2008-06/openpdfs/bairavasundaram.pdf )

One really interesting bit in there I'd never seen before is that they find real data that supports the stand that enterprise drives are significantly more reliable than consumer ones. While general failure rates aren't that different, "SATA disks have an order of magnitude higher probability of developing checksum mismatches than Fibre Channel disks. We find that 0.66% of SATA disks develop at least one mismatch during the first 17 months in the field, whereas only 0.06% of Fibre Channel disks develop a mismatch during that time."

--
* Greg Smith [EMAIL PROTECTED] http://www.gregsmith.com Baltimore, MD

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Reply via email to