On 04.03.2013 22:40, Jeff Davis wrote:
Is there any reason why we can't have both postgres and filesystem checksums?
Of course not. But if we can get away without checksums in Postgres, that's better, because then we don't need to maintain that feature in Postgres. If the patch gets committed, it's not mission accomplished. There will be discussion and need for further development on things like what to do if you get a checksum failure, patches to extend the checksums to cover things like the clog and other non-data files and so forth. And it's an extra complication that will need to be taken into account when developing other new features; in particular, hint bit updates need to write a WAL record. Even if you have all the current hint bits covered, it's an extra hurdle for future patches that might want to have hint bits in e.g new index access methods.
The same user might not want both (or might, if neither are entirely trustworthy yet), but I think it's too early to declare one as the "right" solution and the other not. Even with btrfs stable, I pointed out a number of reasons users might not want it, and reasons that the project should not depend on it.
The PostgreSQL project would not be depending on it, any more than the project depends on filesystem snapshots for backup purposes, or the OS memory manager for caching.
Numbers are always nice, but it takes a lot of effort to come up with them. What kind of numbers are you looking for, and how *specifically* will those numbers affect the decision?
Benchmark of vanilla PostgreSQL, PostgreSQL + this patch, and PostgreSQL running on btrfs or ZFS with data checksums enabled. DBT-2 might be a good candidate, as it's I/O heavy. That would be a good general test; in addition it would be good to see a benchmark of the worst case scenario for the fragmentation you're expecting to see on btrfs, as well as a worst case scenario for the extra WAL traffic with the patch.
If btrfs with checksums is 10% slower than ext4 with postgres checksums, does that mean we should commit the postgres checksums?
In my opinion, a 10% gain would not be worth it, and we should not commit in that case.
On the other side of the coin, if btrfs with checksums is exactly the same speed as ext4 with no postgres checksums (i.e. checksums are free if we use btrfs), does that mean postgres checksums should be rejected?
Yes, I think so. I'm sure at least some others will disagree; Greg already made it quite clear that he doesn't care how the performance of this compares with btrfs.
- Heikki -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
