On 2017-09-01 06:21, ein wrote:
Very comprehensive, thank you. I was asking because I'd like to learn how really random writes by VM affects BTRFS (vs XFS,Ext4) performance and try to develop some workaround to reduce/prevent it while having csums, cow (snapshots) and compression.
I've actually done some significant testing of VM's using BTRFS as backing storage, and would suggest the following:
1. Use raw images in sparse files, and expose them to the VM like SSD's (so that DISCARD support works). Assuming your guests actually support DISCARD commands, this will save you just as much space as using allocate-on-demand formats like qcow2 or VDI, while providing access patterns that BTRFS is a bit better at dealing with. There are a couple of specific caveats to doing this however, namely that you can't use the regular VirtIO block device when using QEMU, and have to use VirtIO SCSI instead, and you'll need smarter tools to copy images around (ddrescue is what I would personally suggest, it can handle sparse files correctly, and gives you a nice progress indicator).
2. Provide separate disks to the VM for any data you need to store in them. By keeping this separate from the root filesystem's disk, you reduce what gets fragmented, and also aren't tied to a particular guest OS to the same degree. As a more concrete example, if you're running a database server in a VM, give it a separate disk image for storing the database from the one the root filesystem is on.
3. Run with autodefrag on the host system. Unless you're hitting the disks hard from inside the VM, this will actually do a reasonably good job of handling fragmentation.
4. Make sure the filesystem you're storing the disk images on has extra space. Ideally, I would set things up so that it has enough room to store a second copy of the largest disk image you expect to use. By keeping lots of extra space, you give the allocator in BTRFS more opportunity to arrange things intelligently.
5. Defragmet your disk images regularly (even when using autodefrag), both inside and outside the VM's. Ideally, run a full defrag inside the VM, then shut it down and defragment the disk image outside the VM. Defragmenting inside the VM first will compact free space, which in turn means that defragmenting outside the VM will do a better job, and that future fragmentation will be less likely. In general, I'd suggest doing this at least monthly (possibly a lot more frequently if you're running databases or similar random access workloads in the VM).
6. Avoid using a CoW filesystem inside the VM's. This sounds odd at first, but is actually one of the biggest things you can do to reduce fragmentation. In particular, try to avoid using BTRFS or ZFS on a VM disk that is stored on BTRFS, both of them will make usual fragmentation problems look nonexistent in comparison.
-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
