Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Jürgen Keil
> But: Isn't there an implicit expectation for a space guarantee associated > with a > dataset? In other words, if a dataset has 1GB of data, isn't it natural to > expect to be able to overwrite that space with other > data? Is there such a space guarantee for compressed or cloned zfs? -- This

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Nils Goroll
> No point in trying to preserve a naive mental model that simply can't stand up to reality. I kind of dislike the idea to talk about naiveness here. Being able to give guarantees (in this case: reserve space) can be vital for running critical business applications. Think about the analogy i

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 15:06, Cyril Plisko wrote: > On Tue, Nov 3, 2009 at 10:54 PM, Nils Goroll wrote: >> But: Isn't there an implicit expectation for a space guarantee >> associated >> with a dataset? In other words, if a dataset has 1GB of data, isn't it >> natural to expect to be able to o

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 16:36, Nils Goroll wrote: > > No point in trying to preserve a naive mental model that >> simply can't stand up to reality. > > I kind of dislike the idea to talk about naiveness here. Maybe it was a poor choice of words; I mean something more along the lines of "simpli

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Nils Goroll
Well, then you could have more "logical space" than "physical space" Reconsidering my own question again, it seems to me that the question of space management is probably more fundamental than I had initially thought, and I assume members of the core team will have thought through much of it.

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Nils Goroll
Hi Cyril, But: Isn't there an implicit expectation for a space guarantee associated with a dataset? In other words, if a dataset has 1GB of data, isn't it natural to expect to be able to overwrite that space with other data? One I'd say that expectation is not [always] valid. Assume you have a

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Cyril Plisko
On Tue, Nov 3, 2009 at 10:54 PM, Nils Goroll wrote: > Now to the more general question: If all datasets of a pool contained the > same data and got de-duped, the sums of their "used" space still seems to be > limited by the "locical" pool size, as we've seen in examples given by > Jürgen and other

Re: [zfs-discuss] ZFS dedup accounting & reservations

2009-11-03 Thread Nils Goroll
Hi David, simply can't stand up to reality. I kind of dislike the idea to talk about naiveness here. Maybe it was a poor choice of words; I mean something more along the lines of "simplistic". The point is, "space" is no longer as simple a concept as it was 40 years ago. Even without dedupl

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Jürgen Keil
> Well, then you could have more "logical space" than > "physical space", and that would be extremely cool, I think we already have that, with zfs clones. I often clone a zfs onnv workspace, and everything is "deduped" between zfs parent snapshot and clone filesystem. The clone (initially) needs

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread David Dyer-Bennet
On Tue, November 3, 2009 10:32, Bartlomiej Pelc wrote: > Well, then you could have more "logical space" than "physical space", and > that would be extremely cool, but what happens if for some reason you > wanted to turn off dedup on one of the filesystems? It might exhaust all > the pool's space t

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Bartlomiej Pelc
Well, then you could have more "logical space" than "physical space", and that would be extremely cool, but what happens if for some reason you wanted to turn off dedup on one of the filesystems? It might exhaust all the pool's space to do this. I think good idea would be another pool's/filesyst

Re: [zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Anurag Agarwal
Hi, It looks interesting problem. Would it help if as ZFS detects dedup blocks, it can start increasing effective size of pool. It will create an anomaly with respect to total disk space, but it will still be accurate from each file system usage point of view. Basically, dedup is at block level,

[zfs-discuss] ZFS dedup accounting

2009-11-03 Thread Nils Goroll
Hi Eric and all, Eric Schrock wrote: On Nov 3, 2009, at 6:01 AM, Jürgen Keil wrote: I think I'm observing the same (with changeset 10936) ... # mkfile 2g /var/tmp/tank.img # zpool create tank /var/tmp/tank.img # zfs set dedup=on tank # zfs create tank/foobar This has to do wit