On Fri, Apr 7, 2017 at 7:50 AM, Austin S. Hemmelgarn <ahferro...@gmail.com> wrote:
> If you care about both performance and data safety, I would suggest using > BTRFS raid1 mode on top of LVM or MD RAID0 together with having good backups > and good monitoring. Statistically speaking, catastrophic hardware failures > are rare, and you'll usually have more than enough warning that a device is > failing before it actually does, so provided you keep on top of monitoring > and replace disks that are showing signs of impending failure as soon as > possible, you will be no worse off in terms of data integrity than running > ext4 or XFS on top of a LVM or MD RAID10 volume. Depending on the workload, and what replication is being used by Ceph above this storage stack, it might make make more sense to do something like three lvm/md raid5 arrays, and then Btrfs single data, raid1 metadata, across those three raid5s. That's giving up only three drives to parity rather than 1/2 the drives, and rebuild time is shorter than losing one drive in a raid0 array. If this is one ceph host, then it might make sense to split the drives up so there are two storage bricks using ceph replication between them for the equivalent of raid1. One brick can do Btrfs on LVM/md raid5, call it brick A. The other brick can do XFS on LVM/md linear, call it brick B. The advantage there is the different bricks are going to have faster commit to stable media times with a mixed workload. The Btrfs on raid5 brick will do better with sequential reads and writes. The XFS on linear will do better with metadata heavy reads and writes. There's probably some Ceph tuning where you can point certain workloads to particular volumes, where those volumes are backed by different priorities to the underlying storage. So you'd setup ceph volume "mail" to be backed in order by brick B then A. Not very well known but XFS will parallelize across drives in a linear/concat arrangement, it's quite useful for e.g. busy mail servers. -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
