Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-27 Thread Jason Dillaman
If I understand correctly you want to look at how many “guest filesystem block size” blocks there are that are empty? This might not be that precise because we do not discard blocks inside the guests, but if you tell me how to gather this - I can certainly try that. I’m not sure if my bash-fu

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Jason Dillaman
Hi all, I am looking for a way to alleviate the overhead of RBD snapshots/clones for some time. In our scenario there are a few “master” volumes that contain production data, and are frequently snapshotted and cloned for dev/qa use. Those snapshots/clones live for a few days to a few weeks

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Haomai Wang
On Fri, Jul 24, 2015 at 11:55 PM, Jason Dillaman dilla...@redhat.com wrote: Hi all, I am looking for a way to alleviate the overhead of RBD snapshots/clones for some time. In our scenario there are a few “master” volumes that contain production data, and are frequently snapshotted and cloned

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Jan Schermer
Hello, If I understand correctly you want to look at how many “guest filesystem block size” blocks there are that are empty? This might not be that precise because we do not discard blocks inside the guests, but if you tell me how to gather this - I can certainly try that. I’m not sure if my

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Reistlin
No thanks at all. I think about ZFS deduplication in a slightly different aspect of using snapshots. We determined, that platter HDD work better with big object size. But it cause big performance overhead with snapshots. For example, you have 32Mb block size. And you have image snapshot. If

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Reistlin
Hi! Did you try ZFS and deduplication mechanism? It could radically decrease writes while COW. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-24 Thread Jan Schermer
We use ZFS for other purposes and deduplication is overrated - it is quite useful with big block sizes (and assuming your data don’t “shift” in the blocks), but you can usually achieve much higher space savings with compression - and it usually is faster, too :-) You need lots and lots of RAM

Re: [ceph-users] Best method to limit snapshot/clone space overhead

2015-07-23 Thread Josh Durgin
On 07/23/2015 06:31 AM, Jan Schermer wrote: Hi all, I am looking for a way to alleviate the overhead of RBD snapshots/clones for some time. In our scenario there are a few “master” volumes that contain production data, and are frequently snapshotted and cloned for dev/qa use. Those