[zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Nils Goroll
Hi Eric and all, Eric Schrock wrote: On Nov 3, 2009, at 6:01 AM, Jürgen Keil wrote: I think I'm observing the same (with changeset 10936) ... # mkfile 2g /var/tmp/tank.img # zpool create tank /var/tmp/tank.img # zfs set dedup=on tank # zfs create tank/foobar This has to do

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Anurag Agarwal
Hi, It looks interesting problem. Would it help if as ZFS detects dedup blocks, it can start increasing effective size of pool. It will create an anomaly with respect to total disk space, but it will still be accurate from each file system usage point of view. Basically, dedup is at block

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Bartlomiej Pelc
Well, then you could have more logical space than physical space, and that would be extremely cool, but what happens if for some reason you wanted to turn off dedup on one of the filesystems? It might exhaust all the pool's space to do this. I think good idea would be another

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 10:32, Bartlomiej Pelc wrote: Well, then you could have more logical space than physical space, and that would be extremely cool, but what happens if for some reason you wanted to turn off dedup on one of the filesystems? It might exhaust all the pool's space to do

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread Nils Goroll
Well, then you could have more logical space than physical space Reconsidering my own question again, it seems to me that the question of space management is probably more fundamental than I had initially thought, and I assume members of the core team will have thought through much of it. I

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread Jürgen Keil
But: Isn't there an implicit expectation for a space guarantee associated with a dataset? In other words, if a dataset has 1GB of data, isn't it natural to expect to be able to overwrite that space with other data? Is there such a space guarantee for compressed or cloned zfs? -- This

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread Cyril Plisko
On Tue, Nov 3, 2009 at 10:54 PM, Nils Goroll sl...@schokola.de wrote: Now to the more general question: If all datasets of a pool contained the same data and got de-duped, the sums of their used space still seems to be limited by the locical pool size, as we've seen in examples given by Jürgen

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 16:36, Nils Goroll wrote: No point in trying to preserve a naive mental model that simply can't stand up to reality. I kind of dislike the idea to talk about naiveness here. Maybe it was a poor choice of words; I mean something more along the lines of simplistic.

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Jürgen Keil
Well, then you could have more logical space than physical space, and that would be extremely cool, I think we already have that, with zfs clones. I often clone a zfs onnv workspace, and everything is deduped between zfs parent snapshot and clone filesystem. The clone (initially) needs no

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 15:06, Cyril Plisko wrote: On Tue, Nov 3, 2009 at 10:54 PM, Nils Goroll sl...@schokola.de wrote: But: Isn't there an implicit expectation for a space guarantee associated with a dataset? In other words, if a dataset has 1GB of data, isn't it natural to expect to be

Re: [zfs-discuss] ZFS dedup accounting reservations

2009-11-03 Thread Nils Goroll
Hi David, simply can't stand up to reality. I kind of dislike the idea to talk about naiveness here. Maybe it was a poor choice of words; I mean something more along the lines of simplistic. The point is, space is no longer as simple a concept as it was 40 years ago. Even without