Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On 17/09/11 00:09, Vick Khera wrote: On Tue, Sep 13, 2011 at 9:15 PM, Toby Corkindale toby.corkind...@strategicdata.com.au wrote: However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! Did you do any tuning to ZFS? There are many tweaks to it, like putting a cache disk in front of it, or moving the logs to SSD and such. I haven't run any produciton DBs on ZFS yet, but it sure is tempting. The speed penalty for the features it gives you (snapshots, robust against power fails, etc.) is worth the tradeoff. No, I didn't do that kind of tuning - agreed, it'd improve the performance. But then putting an SSD in the mix and storing journals on it would have improved the performance of XFS and ext4 as well.. I'll re-run the tests again in the future, no doubt, and hopefully I'll have a spare SSD by then. Also maybe I'll have learnt more about ZFS; I'm a bit of a noob at the moment. I agree that ZFS does seem to offer some rather nice features though! I'm tempted to start using it on my personal server now; although I'll be leaving it for some time before considering using it in production at work. Cheers, Toby -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On Tue, Sep 13, 2011 at 9:15 PM, Toby Corkindale toby.corkind...@strategicdata.com.au wrote: However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! Did you do any tuning to ZFS? There are many tweaks to it, like putting a cache disk in front of it, or moving the logs to SSD and such. I haven't run any produciton DBs on ZFS yet, but it sure is tempting. The speed penalty for the features it gives you (snapshots, robust against power fails, etc.) is worth the tradeoff. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On Wed, Sep 14, 2011 at 11:18 PM, Toby Corkindale toby.corkind...@strategicdata.com.au wrote: The zpool was created against an LVM logical volume (which was the same one used for all the filesystems measured in the tests). That LV was itself part of a volume group that was striped over three disks (Western Digital WD1003FBYX). So you're throwing in the complication of a logical volume manager interfering with zfs. If you put ZFS on three separate drives, it has a better chance of optimizing its operations. -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On 14/09/11 12:56, Andy Colson wrote: On 09/13/2011 08:15 PM, Toby Corkindale wrote: Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). [snip] Did you test unplugging the power cable in the middle of a test to see which would come back up? Heh, no, but it'd be interesting to see.. I wonder if turning a virtual machine off has the same effect? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On 14/09/2011 09:30, Toby Corkindale wrote: On 14/09/11 12:56, Andy Colson wrote: On 09/13/2011 08:15 PM, Toby Corkindale wrote: Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). [snip] Did you test unplugging the power cable in the middle of a test to see which would come back up? Heh, no, but it'd be interesting to see.. I wonder if turning a virtual machine off has the same effect? No, never the exact same effect. There are two things to consider: if/when/how the OS flushes the data to the hardware and if/when/how the hardware flushes the data to physical storage. You can simulate only the failure of the first part with a virtual machine, but not the second (unless you bring down the VM host...). -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
Can you go into some more detail on how you set up ZFS on these systems? Robert Treat conjecture: xzilla.net consulting: omniti.com On Tue, Sep 13, 2011 at 10:56 PM, Andy Colson a...@squeakycode.net wrote: On 09/13/2011 08:15 PM, Toby Corkindale wrote: Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). Since then the kernel-level version of ZFS became usable, and there have been improvements to btrfs, and no doubt various updates in the Linux kernel and PostgreSQL that should help performance. I ran the tests on Ubuntu 11.04 with Pg 9.0 first, then upgraded the system to Ubuntu 11.10 (beta) with Pg 9.1 and ran them again. The latter combination showed a considerable performance improvement overall - although I didn't investigate to find out whether this was due to kernel improvements, postgres improvements, or virtio improvements. The results are measured in transactions-per-second, with higher numbers being better. Results: ext4 (data=writeback,relatime): natty: 248 oneiric: 297 ext4 (data=writeback,relatime,nobarrier): natty: didn't test oneiric: 1409 XFS (relatime): natty: didn't test oneiric: 171 btrfs (relatime): natty: 61.5 oneiric: 91 btrfs (relatime,nodatacow): natty: didn't test oneiric: 128 ZFS (defaults): natty: 171 oneiric: 996 Conclusion: Last time I ran these tests, xfs and ext4 pulled very similar results, and both were miles ahead of btrfs. This time around, ext4 has managed to get a significantly faster result than xfs. However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! I'm not sure how it achieved this, and whether we're losing some kind of data protection (eg. like the barrier options in XFS and ext4). If ext4 has barriers disabled, it surpasses even ZFSs high score. Oddly, ZFS performed wildly differently on ubuntu 11.04 vs 11.10b. I can't explain this. Any ideas? Cheers, Toby Did you test unplugging the power cable in the middle of a test to see which would come back up? -Andy -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
Dne 14.9.2011 03:15, Toby Corkindale napsal(a): Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). Since then the kernel-level version of ZFS became usable, and there have been improvements to btrfs, and no doubt various updates in the Linux kernel and PostgreSQL that should help performance. I ran the tests on Ubuntu 11.04 with Pg 9.0 first, then upgraded the system to Ubuntu 11.10 (beta) with Pg 9.1 and ran them again. The latter combination showed a considerable performance improvement overall - although I didn't investigate to find out whether this was due to kernel improvements, postgres improvements, or virtio improvements. The results are measured in transactions-per-second, with higher numbers being better. Results: ext4 (data=writeback,relatime): natty: 248 oneiric: 297 ext4 (data=writeback,relatime,nobarrier): natty: didn't test oneiric: 1409 XFS (relatime): natty: didn't test oneiric: 171 btrfs (relatime): natty: 61.5 oneiric: 91 btrfs (relatime,nodatacow): natty: didn't test oneiric: 128 ZFS (defaults): natty: 171 oneiric: 996 Conclusion: Last time I ran these tests, xfs and ext4 pulled very similar results, and both were miles ahead of btrfs. This time around, ext4 has managed to get a significantly faster result than xfs. However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! I'm not sure how it achieved this, and whether we're losing some kind of data protection (eg. like the barrier options in XFS and ext4). If ext4 has barriers disabled, it surpasses even ZFSs high score. Oddly, ZFS performed wildly differently on ubuntu 11.04 vs 11.10b. I can't explain this. Any ideas? Those results look very suspisicous I guess. Unless we know all the details about the fs options, pgbench runs and system (drives, cache, controller) it's very difficult to say anything. For example the ZFS results seem really strange - I've done a quite thorough benchmark of file systems last month (http://bit.ly/oEdkSH) and I really don't think the differences can be that huge under the same conditions - increase from 171 to 996 is ridiculous. The only explanation I can come up with is that oneiric uses significantly weaker ZFS options (e.g. disabled write barriers) and there's a controller with write cache. But that also means you should not compare ZFS to plain ext4 (297 tps) but to ext4 with write barriers disabled (1409 tps). That changes the results a bit, because instead of being 3x faster, ZFS is about 30% slower. Tomas -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On 15/09/11 02:46, Robert Treat wrote: Can you go into some more detail on how you set up ZFS on these systems? I'm afraid my knowledge of ZFS is rather weak compared to the other filesystems - all I really did was zpool create followed by zfs create, using all the defaults. The zpool was created against an LVM logical volume (which was the same one used for all the filesystems measured in the tests). That LV was itself part of a volume group that was striped over three disks (Western Digital WD1003FBYX). I'm happy to re-run the benchmark with different ZFS options if you can suggest some. Unfortunately I can't easily bypass the LVM layer here, as the disks involved are fully committed to the volume group and I don't want to rebuild the machine. Cheers, Toby On Tue, Sep 13, 2011 at 10:56 PM, Andy Colsona...@squeakycode.net wrote: On 09/13/2011 08:15 PM, Toby Corkindale wrote: Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). Since then the kernel-level version of ZFS became usable, and there have been improvements to btrfs, and no doubt various updates in the Linux kernel and PostgreSQL that should help performance. I ran the tests on Ubuntu 11.04 with Pg 9.0 first, then upgraded the system to Ubuntu 11.10 (beta) with Pg 9.1 and ran them again. The latter combination showed a considerable performance improvement overall - although I didn't investigate to find out whether this was due to kernel improvements, postgres improvements, or virtio improvements. The results are measured in transactions-per-second, with higher numbers being better. Results: ext4 (data=writeback,relatime): natty: 248 oneiric: 297 ext4 (data=writeback,relatime,nobarrier): natty: didn't test oneiric: 1409 XFS (relatime): natty: didn't test oneiric: 171 btrfs (relatime): natty: 61.5 oneiric: 91 btrfs (relatime,nodatacow): natty: didn't test oneiric: 128 ZFS (defaults): natty: 171 oneiric: 996 Conclusion: Last time I ran these tests, xfs and ext4 pulled very similar results, and both were miles ahead of btrfs. This time around, ext4 has managed to get a significantly faster result than xfs. However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! I'm not sure how it achieved this, and whether we're losing some kind of data protection (eg. like the barrier options in XFS and ext4). If ext4 has barriers disabled, it surpasses even ZFSs high score. Oddly, ZFS performed wildly differently on ubuntu 11.04 vs 11.10b. I can't explain this. Any ideas? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
Re: [GENERAL] PostgreSQL benchmarked on XFS vs ZFS vs btrfs vs ext4
On 09/13/2011 08:15 PM, Toby Corkindale wrote: Hi, Some months ago, I ran some (probably naive) benchmarks looking at how pgbench performed on an identical system with differing filesystems. (on Linux). Since then the kernel-level version of ZFS became usable, and there have been improvements to btrfs, and no doubt various updates in the Linux kernel and PostgreSQL that should help performance. I ran the tests on Ubuntu 11.04 with Pg 9.0 first, then upgraded the system to Ubuntu 11.10 (beta) with Pg 9.1 and ran them again. The latter combination showed a considerable performance improvement overall - although I didn't investigate to find out whether this was due to kernel improvements, postgres improvements, or virtio improvements. The results are measured in transactions-per-second, with higher numbers being better. Results: ext4 (data=writeback,relatime): natty: 248 oneiric: 297 ext4 (data=writeback,relatime,nobarrier): natty: didn't test oneiric: 1409 XFS (relatime): natty: didn't test oneiric: 171 btrfs (relatime): natty: 61.5 oneiric: 91 btrfs (relatime,nodatacow): natty: didn't test oneiric: 128 ZFS (defaults): natty: 171 oneiric: 996 Conclusion: Last time I ran these tests, xfs and ext4 pulled very similar results, and both were miles ahead of btrfs. This time around, ext4 has managed to get a significantly faster result than xfs. However we have a new contender - ZFS performed *extremely* well on the latest Ubuntu setup - achieving triple the performance of regular ext4! I'm not sure how it achieved this, and whether we're losing some kind of data protection (eg. like the barrier options in XFS and ext4). If ext4 has barriers disabled, it surpasses even ZFSs high score. Oddly, ZFS performed wildly differently on ubuntu 11.04 vs 11.10b. I can't explain this. Any ideas? Cheers, Toby Did you test unplugging the power cable in the middle of a test to see which would come back up? -Andy -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general
