[ceph-users] ZFS or BTRFS for performance?

2016-03-18 Thread Schlacta, Christ
Insofar as I've been able to tell, both BTRFS and ZFS provide similar capabilities back to CEPH, and both are sufficiently stable for the basic CEPH use case (Single disk -> single mount point), so the question becomes this: Which actually provides better performance? Which is the more highly opti

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-18 Thread Lionel Bouton
Hi, Le 18/03/2016 20:58, Mark Nelson a écrit : > FWIW, from purely a performance perspective Ceph usually looks pretty > fantastic on a fresh BTRFS filesystem. In fact it will probably > continue to look great until you do small random writes to large > objects (like say to blocks in an RBD volum

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Heath Albritton
Neither of these file systems is recommended for production use underlying an OSD. The general direction for ceph is to move away from having a file system at all. That effort is called "bluestore" and is supposed to show up in the jewel release. -H > On Mar 18, 2016, at 11:15, Schlacta, Chr

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Schlacta, Christ
On Mar 18, 2016 4:31 PM, "Lionel Bouton" > > Will bluestore provide the same protection against bitrot than BTRFS? > Ie: with BTRFS the deep-scrubs detect inconsistencies *and* the OSD(s) > with invalid data get IO errors when trying to read corrupted data and > as such can't be used as the source

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Heath Albritton
If you google "ceph bluestore" you'll be able to find a couple slide decks on the topic. One of them by Sage is easy to follow without the benefit of the presentation. There's also the " Redhat Ceph Storage Roadmap 2016" deck. In any case, bluestore is not intended to address bitrot. Given th

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Mark Nelson
FWIW, from purely a performance perspective Ceph usually looks pretty fantastic on a fresh BTRFS filesystem. In fact it will probably continue to look great until you do small random writes to large objects (like say to blocks in an RBD volume). Then COW starts fragmenting the objects into ob

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Lindsay Mathieson
On 20/03/2016 3:38 AM, Heath Albritton wrote: Ceph protects against bitrot at a much higher level by validating the checksum of the entire placement group during a deep scrub. Ceph has checksums? I didn't think it did. Its my understanding that it just compares blocks between replications

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Lionel Bouton
Le 19/03/2016 18:38, Heath Albritton a écrit : > If you google "ceph bluestore" you'll be able to find a couple slide > decks on the topic. One of them by Sage is easy to follow without the > benefit of the presentation. There's also the " Redhat Ceph Storage > Roadmap 2016" deck. > > In any case

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Nmz
Yes, I`m missing protection from Ceph too. http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-February/007680.html ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-19 Thread Christian Balzer
Hello, On Sun, 20 Mar 2016 00:45:47 +0100 Lionel Bouton wrote: > Le 19/03/2016 18:38, Heath Albritton a écrit : > > If you google "ceph bluestore" you'll be able to find a couple slide > > decks on the topic. One of them by Sage is easy to follow without the > > benefit of the presentation. Th

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-20 Thread Francois Lafont
Hello, On 20/03/2016 04:47, Christian Balzer wrote: > That's not protection, that's an "uh-oh, something is wrong, you better > check it out" notification, after which you get to spend a lot of time > figuring out which is the good replica In fact, I have never been confronted to this case so f

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-20 Thread Lionel Bouton
Hi, Le 20/03/2016 15:23, Francois Lafont a écrit : > Hello, > > On 20/03/2016 04:47, Christian Balzer wrote: > >> That's not protection, that's an "uh-oh, something is wrong, you better >> check it out" notification, after which you get to spend a lot of time >> figuring out which is the good repl

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-20 Thread Mike Almateia
18-Mar-16 21:15, Schlacta, Christ пишет: Insofar as I've been able to tell, both BTRFS and ZFS provide similar capabilities back to CEPH, and both are sufficiently stable for the basic CEPH use case (Single disk -> single mount point), so the question becomes this: Which actually provides better

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-20 Thread Schlacta, Christ
What do you use as an interconnect between your osds, and your clients? On Mar 20, 2016 11:39 AM, "Mike Almateia" wrote: > 18-Mar-16 21:15, Schlacta, Christ пишет: > >> Insofar as I've been able to tell, both BTRFS and ZFS provide similar >> capabilities back to CEPH, and both are sufficiently st

Re: [ceph-users] ZFS or BTRFS for performance?

2016-03-22 Thread Mike Almateia
20-Mar-16 23:23, Schlacta, Christ пишет: What do you use as an interconnect between your osds, and your clients? Two Mellanox 10Gb SFP NIC dual port each = 4 x 10Gbit/s ports on each server. On servers each 2 ports bonded, so we have 2 bond for Cluster net and Storage net. Clients servers