On 02/22/2016 03:21 PM, Stefan Priebe wrote: > Hello, > > is there any chance or hack to work with a bigger cluster size for the > drive backup job? > > See: > http://git.qemu.org/?p=qemu.git;a=blob;f=block/backup.c;h=16105d40b193be9bb40346027bdf58e62b956a96;hb=98d2c6f2cd80afaa2dc10091f5e35a97c181e4f5 > > > This is very slow with ceph - may be due to the 64k block size. I would > like to check whether this is faster with cephs native block size of 4mb. > > Greets, > Stefan >
It's hardcoded to 64K at the moment, but I am checking in a patch to round up the cluster size to be the bigger of (64k, $target_cluster_size) in order to make sure that incremental backups in particular never copy a fraction of a cluster. As a side-effect, the same round-up will happen for all modes (sync=top,none,full). If QEMU is aware of the target cluster size of 4MB, this would immediately jump the copy-size up to 4MB clusters for you. See: https://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg02839.html Otherwise, after my trivial fix, you should find cluster_size to be a mutable concept and perhaps one that you could introduce a runtime parameter for if you could convince the necessary parties that it's needed in the API. You'd have to be careful in the case of incremental that all the various cluster sizes work well together: - Granularity of bitmap (Defaults to cluster size of source, or 64K if unknown. can be configured to be arbitrarily larger.) - Cluster size of source file (For qcow2, defaults to 64k) - Cluster size of target file - Cluster size of backup routine (Currently always 64K) I think that LCM(source_cluster_size, target_cluster_size, backup_cluster_size) = MAX(A, B, C) will always be a safe minimum. Bitmap granularity is more flexible, and it is more efficient when the bitmap granularity matches the backup cluster size, but it can cope with mismatches. If the bitmap is smaller or larger than the backup cluster size, it generally means that more data that is clean is going to be transferred across the pipe. (Hmm, I wonder if it's worth checking in code to adjust a bitmap granularity after it has been created so people can easily experiment with performance tweaking here.) Let me know if any of my ramble sounds interesting :) --John
