I am running a single btrfs RAID10 volume of eight LUKS devices, each using a 2TB SATA hard drive as a backing store. The SATA drives are a mixture of Seagate and Western Digital drives, some with RPMs ranging from 5400 to 7200. Each seems to individually performance test where I would expect for drives of this caliber. They are all attached to an LSI PCIe SAS controller and configured in JBOD.
I have a relatively beefy quad core Xeon CPU that supports AES-NI and don't think LUKS is my bottleneck. Here's some info from the resulting filesystem: btrfs fi df /storage Data, RAID10: total=6.30TiB, used=6.29TiB System, RAID10: total=8.00MiB, used=560.00KiB Metadata, RAID10: total=9.00GiB, used=7.64GiB GlobalReserve, single: total=512.00MiB, used=0.00B In general I see good performance, especially read performance which is enough to regularly saturate my gigabit network when copying files from this host via samba. Reads are definitely taking advantage of the multiple copies of data available and spreading the load among all drives. Writes aren't quite as rosy, however. When writing files using dd like in this example: dd if=/dev/zero of=tempfile bs=1M count=10240 conv=fdatasync,notrun c status=progress And running a command like: iostat -m 1 to monitor disk I/O, writes seem to only focus on one of the eight disks at a time, moving from one drive to the next. This results in a sustained 55-90 MB/sec throughput depending on which disk is being written to (remember, some have faster spindle speed than others). Am I wrong to expect btrfs' RAID10 mode to write to multiple disks simultaneously and to break larger writes into smaller stripes across my four pairs of disks? I had trouble identifying whether btrfs RAID10 is writing (64K?) stripes or (1GB?) blocks to disk in this mode. The latter might make more sense based upon what I'm seeing? Anything else I should be trying to narrow down the bottleneck? Thanks! -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
