Re: Disappearing available space with ZFS...what am I missing?
On 2010-09-29 07:56, Aaron wrote: I've created a ZFS pool with zpool create tank raidz ada0 ada1 ada2 ada3, and then I add some additional mountpoints (I think they're called) using zfs create tank/storage, etc. In zpool list, I see the pool with 3.62T available. With df -h, I see 2.4T available for tank, and tank/storage. When I first created tank, it had the 3.62T available as I expected. What am I missing? I do have compression set to gzip-9 on tank which gets inherited like I want, don't know if that would affect anything. --Aaron There's nothing wrong here that I can see, you just have to make a distinction between the zfs pool and the filesystems within the pool and I agree it can be confusing at first. The numbers suggest you are using 4 x 1TB (base 10 TB) drives? That equals 3.7TiB (base 2 TB) which is the unit zpool/zfs uses. This is the total amount of space available to the pool and includes all space on all drives in the pool. Nothing strange so far. Now, since you've told zpool to create filesystems within the pool using raidz, the filesystems will have 25% less space available since this space is used for parity data. So a filesystem using the whole pool will report having 3.7 * 0.75 = 2.7TiB available which is in agreement with your numbers. A raidz filesystem will always lose 1 disk worth of space and will never report that space as available to you since it will always be occupied with parity data. The pool on the other hand doesn't make a distinction, in this case anyway, between user data and parity data so zpool will always report what's actually unallocated on all your physical drives in the pool. For every GiB you allocate in the filesystem you will allocate 1.33GiB in the pool since that includes parity data. "zfs list" and "df -h" are your best friends to find out how much space is available for your files. Don't bother about "zpool list". Regards Morgan ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Disappearing available space with ZFS...what am I missing?
On 09/29/2010 12:56 AM, Aaron wrote: > I've created a ZFS pool with zpool create tank raidz ada0 ada1 ada2 > ada3, and then I add some additional mountpoints (I think they're > called) using zfs create tank/storage, etc. In zpool list, I see the > pool with 3.62T available. With df -h, I see 2.4T available for tank, > and tank/storage. When I first created tank, it had the 3.62T > available as I expected. What am I missing? I do have compression set > to gzip-9 on tank which gets inherited like I want, don't know if that > would affect anything. zpool list shows raw space, and zfs list (and, to a certain extent, df) shows effective capacity. You've configured a raidz, which chews up extra space to store the redundant parity information; thus, for a 4-disk raidz, 1GB of stored data will use up about 1.33GB of raw space. Since effective capacity in ZFS is so variable, due to things like ditto blocks, raidz, compression, and (soon to be added) dedup, the numbers provided by df no longer make much sense. This blog posting can provide further elaboration as to why df doesn't work for advanced file storage: http://www.c0t0d0s0.org/archives/6168-df-considered-problematic.html -- Fuzzy love, -CyberLeo Technical Administrator CyberLeo.Net Webhosting http://www.CyberLeo.Net Furry Peace! - http://.fur.com/peace/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Disappearing available space with ZFS...what am I missing?
On 29 September 2010 10:12, Morgan Wesström wrote: > On 2010-09-29 07:56, Aaron wrote: > >> I've created a ZFS pool with zpool create tank raidz ada0 ada1 ada2 >> ada3, and then I add some additional mountpoints (I think they're >> called) using zfs create tank/storage, etc. In zpool list, I see the >> pool with 3.62T available. With df -h, I see 2.4T available for tank, >> and tank/storage. When I first created tank, it had the 3.62T >> available as I expected. What am I missing? I do have compression set >> to gzip-9 on tank which gets inherited like I want, don't know if that >> would affect anything. >> >> --Aaron >> > > There's nothing wrong here that I can see, you just have to make a > distinction between the zfs pool and the filesystems within the pool and I > agree it can be confusing at first. > > The numbers suggest you are using 4 x 1TB (base 10 TB) drives? That equals > 3.7TiB (base 2 TB) which is the unit zpool/zfs uses. This is the total > amount of space available to the pool and includes all space on all drives > in the pool. Nothing strange so far. > > Now, since you've told zpool to create filesystems within the pool using > raidz, the filesystems will have 25% less space available since this space > is used for parity data. So a filesystem using the whole pool will report > having 3.7 * 0.75 = 2.7TiB available which is in agreement with your > numbers. A raidz filesystem will always lose 1 disk worth of space and will > never report that space as available to you since it will always be occupied > with parity data. > > The pool on the other hand doesn't make a distinction, in this case anyway, > between user data and parity data so zpool will always report what's > actually unallocated on all your physical drives in the pool. For every GiB > you allocate in the filesystem you will allocate 1.33GiB in the pool since > that includes parity data. "zfs list" and "df -h" are your best friends to > find out how much space is available for your files. Don't bother about > "zpool list". > > Regards > Morgan > > ___ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to " > freebsd-questions-unsubscr...@freebsd.org" > It gets even more hairy when you start adding in reservsions, quotas, and compression. Slap dedup on top of that and you get magically growing fs according to df 8) ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
Re: Disappearing available space with ZFS...what am I missing?
On Wed, Sep 29, 2010 at 03:14, krad wrote: > > > On 29 September 2010 10:12, Morgan Wesström > wrote: >> >> On 2010-09-29 07:56, Aaron wrote: >>> >>> I've created a ZFS pool with zpool create tank raidz ada0 ada1 ada2 >>> ada3, and then I add some additional mountpoints (I think they're >>> called) using zfs create tank/storage, etc. In zpool list, I see the >>> pool with 3.62T available. With df -h, I see 2.4T available for tank, >>> and tank/storage. When I first created tank, it had the 3.62T >>> available as I expected. What am I missing? I do have compression set >>> to gzip-9 on tank which gets inherited like I want, don't know if that >>> would affect anything. >>> >>> --Aaron >> >> There's nothing wrong here that I can see, you just have to make a >> distinction between the zfs pool and the filesystems within the pool and I >> agree it can be confusing at first. >> >> The numbers suggest you are using 4 x 1TB (base 10 TB) drives? That equals >> 3.7TiB (base 2 TB) which is the unit zpool/zfs uses. This is the total >> amount of space available to the pool and includes all space on all drives >> in the pool. Nothing strange so far. >> >> Now, since you've told zpool to create filesystems within the pool using >> raidz, the filesystems will have 25% less space available since this space >> is used for parity data. So a filesystem using the whole pool will report >> having 3.7 * 0.75 = 2.7TiB available which is in agreement with your >> numbers. A raidz filesystem will always lose 1 disk worth of space and will >> never report that space as available to you since it will always be occupied >> with parity data. >> >> The pool on the other hand doesn't make a distinction, in this case >> anyway, between user data and parity data so zpool will always report what's >> actually unallocated on all your physical drives in the pool. For every GiB >> you allocate in the filesystem you will allocate 1.33GiB in the pool since >> that includes parity data. "zfs list" and "df -h" are your best friends to >> find out how much space is available for your files. Don't bother about >> "zpool list". >> >> Regards >> Morgan >> ___ >> freebsd-questions@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >> To unsubscribe, send any mail to >> "freebsd-questions-unsubscr...@freebsd.org" > > > It gets even more hairy when you start adding in reservsions, quotas, and > compression. Slap dedup on top of that and you get magically growing fs > according to df 8) > > Ahhh...yea. Thanks everyone! I didn't realize that zpool status would show the raw space (so 4x1TB base10 is ~3.7TB base2), and not the available space after the 4-1 RAIDZ usage consumes. So that all makes sense, as well as with reservations, quotas and compression now causing df to not really know what's going on. If I want to get the actual values of available space vs used space, I should use zfs get all and look at the properties there? --Aaron ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "freebsd-questions-unsubscr...@freebsd.org"
