Hi all as part of moving our OpenStack VM instance store from dedicated disks on the physical hosts to a CephFS backed by an SSD pool, we noticed that the files created on CephFS aren't sparse, even though the original files were.
This is on root@s2:~# ls -lhs /var/lib/nova/instances/_base total 63G 750M -rw-r--r-- 1 nova nova 2.0G Jul 10 21:40 1a11de23fe75a210b4da631366513cb7c22ef311 750M -rw-r--r-- 1 libvirt-qemu kvm 10G Jul 10 21:40 1a11de23fe75a210b4da631366513cb7c22ef311_10 … vs root@s2:~# ls -lhs /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311* 1.2G -rw-r--r-- 1 nova nova 1.2G Sep 5 16:56 /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311 10G -rw-r--r-- 1 libvirt-qemu kvm 10G Jul 10 21:40 /mnt/instances/instances/_base/1a11de23fe75a210b4da631366513cb7c22ef311_10 We have used different ways of copying the files (tar and rsync) and specified the sparse options: # rsync -rtvupogS -h /var/lib/nova/instances/ /mnt/instances/instances or # (cd /var/lib/nova/instances ; tar -Svcf - .)|(cd /mnt/instances/instances ; tar Sxpf -) The OSDs we use for this pool are backed by XFS (which has a problem with sparse files, unless one specifies allocation block size options in the mounts) http://serverfault.com/questions/406069/why-are-my-xfs-filesystems-suddenly-consuming-more-space-and-full-of-sparse-file, https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=055388a3188f56676c21e92962fc366ac8b5cb72. We have mounted the XFS partitions for the OSDs with this option, but I assume that this shouldn't impact the way CephFS handles sparse files. I seem to remember that the copying of sparse files worked a couple of months ago (ceph-fs kernel 3.5 on btrfs OSDs), but now we used Kernel 3.10 and recently ceph-fuse to mount the CephFS. Are we doing something wrong, or is this not supported by CephFS? cheers jc -- SWITCH Jens-Christian Fischer, Peta Solutions Werdstrasse 2, P.O. Box, 8021 Zurich, Switzerland phone +41 44 268 15 15, direct +41 44 268 15 71 jens-christian.fisc...@switch.ch http://www.switch.ch http://www.switch.ch/socialmedia
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
