Re: [zfs-discuss] zfs question as to sizes
Eric Schrock <[EMAIL PROTECTED]> wrote on 04/16/2007 05:29:05 PM: > On Mon, Apr 16, 2007 at 05:13:37PM -0500, [EMAIL PROTECTED] wrote: > > > > Why it was considered a valid data column in its current state is > > anyone's guess. > > > > This column is precise and valid. It represents the amount of space > uniquely referenced by the snapshot, and therefore the amount of space > that would be freed were it to be deleted. > The OP's question and my response should indicate that while this number may be precise and valid, it is not useful for most administrators workflow. > The shared space between snapshots, besides being difficult to > calculate, is nearly impossible to enumerate in anything beyond the most > trivial setups. For example, with just snapshots 'a b c d e', you can > have space shared by the following combinations: >a b >a b c >a b c d >a b c d e >b c >b c d >b c d e >c d >c d e >d e It is complex, I will give you that. Most of the accounting needed is already done in dsl's born and dead code. > > Not to mention the space shared with the active filesystem. With dozens > of snapshots, you're talking about hundreds or thousands of > combinations. It's certainly possible to calculate the space used > various snapshot intersections, but presenting it is another matter. > Perhaps you could describe how you would like this information to be > presented in zfs(1M). > Hello Eric, I am not looking for a gigantic gantt chart. Displaying the data in one way that is useful across the board is a nontrivial task -- but coming up with 3 or 4 ways to dive into the data from different perspectives may cover all but the most edge cases. It is not acceptable to ask admins to delete and find out blindly -- we like to be able to plan before we act. This is unacceptable when you get to anything beyond the most trivial of setups. In its current form, we can have more "hidden" allocated storage than reported usage is never a pleasant place to be in. I consider that broke. How is an administrator to know snap 46 -> 48 are taking up 4 tb when you have 2000+ snaps and looking to recover freespace -- zfs list seems to imply we should get along with the 1.2m and 300m listed as "used" for those two snaps? I suggest that the default zfs list should show the space (all of the space) accounted in the last snapshot before the dead list for the block. You treat snapshots as a timeline in code, while discussing the functionality and in usage -- do so in default reporting too. This simple changes gives admins the ability to see large growth spikes/deletes inline. This shows what space would be freed when deleting snaps from the oldest one to X. when a snap is killed off in the middle while you are inspecting the born dead blocks adjust the usage counter to the right or off the books. Change the "unique" listing to be zfs list -o snapunique add other flags such as snapborn and snapdead to show the admin the flow of data when doing zfs list -o snapunique,snapborn,snapdead. snapborn and dead should show the usage born and marked dead on each unique snapshot. third, zfs listsnap to show how much unique space is reserved and would be freed by deleting the snapshots in the list. The data is all there in the born and dead lists, but admins are stuck with a view that completely hides all space reservations that are used across two or more snaps. Put out an RFC on this, I am sure there are many ideas floating around about how the utilization on snaps should or could be reported. I think most everyone understands that doing some forms of reporting are non-trivial. Where we are at now is hiding too much data and revealing data that is a poor/invalid picture of the true state. kindest regards, -Wade ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs question as to sizes
On Mon, Apr 16, 2007 at 05:13:37PM -0500, [EMAIL PROTECTED] wrote: > > Why it was considered a valid data column in its current state is > anyone's guess. > This column is precise and valid. It represents the amount of space uniquely referenced by the snapshot, and therefore the amount of space that would be freed were it to be deleted. The shared space between snapshots, besides being difficult to calculate, is nearly impossible to enumerate in anything beyond the most trivial setups. For example, with just snapshots 'a b c d e', you can have space shared by the following combinations: a b a b c a b c d a b c d e b c b c d b c d e c d c d e d e Not to mention the space shared with the active filesystem. With dozens of snapshots, you're talking about hundreds or thousands of combinations. It's certainly possible to calculate the space used various snapshot intersections, but presenting it is another matter. Perhaps you could describe how you would like this information to be presented in zfs(1M). - Eric -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs question as to sizes
[EMAIL PROTECTED] wrote on 04/16/2007 04:57:43 PM: > one pool is mirror on 300gb dirives and the other is raidz1 on 7 x > 143gb drives. > > I did make clone of my zfs file systems with their snaps and something is not > right, sizes do not match... anyway here is what I have: > > [17:50:32] [EMAIL PROTECTED]: /root > zfs list > NAME USED AVAIL REFER MOUNTPOINT > mypool 272G 1.95G 24.5K /mypool > mypool/d 272G 1.95G 143G /d/d2 > mypool/[EMAIL PROTECTED] 3.72G - 123G - > mypool/[EMAIL PROTECTED] 22.3G - 156G - > mypool/[EMAIL PROTECTED] 23.3G - 161G - > mypool/[EMAIL PROTECTED] 16.1G - 172G - > mypool/[EMAIL PROTECTED] 13.8G - 168G - > mypool/[EMAIL PROTECTED] 15.7G - 168G - > mypool/[EMAIL PROTECTED]192M - 143G - > mypool/[EMAIL PROTECTED]189M - 143G - > mypool/[EMAIL PROTECTED]200M - 143G - > mypool/[EMAIL PROTECTED] 3.93M - 143G - > mypool2463G 474G52K /mypool2 > mypool2/d 318G 474G 168G /mypool2/d > mypool2/[EMAIL PROTECTED] 4.40G - 145G - > mypool2/[EMAIL PROTECTED] 26.1G - 184G - > mypool2/[EMAIL PROTECTED] 27.3G - 189G - > mypool2/[EMAIL PROTECTED] 18.7G - 202G - > mypool2/[EMAIL PROTECTED] 16.1G - 197G - > mypool2/[EMAIL PROTECTED]18.2G - 198G - > mypool2/d3 145G 474G 145G legacy > > see: > mypool/d 272G 1.95G 143G /d/d2 > mypool2/d 318G 474G 168G /mypool2/d > > they are the same copies but their sizes do differ quite a bit, > original is 272G > and the "clone" which I duplicated by zfs send/receive is 318G. Then all the > other snaps also do differ. Why is that difference? Could someone explain how > does it work? > No, snapshot space usage reporting is beyond brain-dead at this point, shared data between snaps is hidden from the reporting and there is no way (short of deleting snapshots) to see how much space they are holding hostage. The ZFS team has said that they are working on providing more detail -- I have not seen anything yet. Why it was considered a valid data column in its current state is anyone's guess -- in one of my servers I have over 7 tb unaccounted for in the zfs listing because of this issue. -Wade ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] zfs question as to sizes
ok, here is what I have: [17:53:35] [EMAIL PROTECTED]: /root > zpool status -v pool: mypool state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirrorONLINE 0 0 0 c1t2d0 ONLINE 0 0 0 c1t3d0 ONLINE 0 0 0 errors: No known data errors pool: mypool2 state: ONLINE scrub: none requested config: NAMESTATE READ WRITE CKSUM mypool2 ONLINE 0 0 0 raidz1ONLINE 0 0 0 c3t0d0 ONLINE 0 0 0 c3t1d0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 c3t4d0 ONLINE 0 0 0 c3t5d0 ONLINE 0 0 0 c3t6d0 ONLINE 0 0 0 errors: No known data errors [17:53:37] [EMAIL PROTECTED]: /root > one pool is mirror on 300gb dirives and the other is raidz1 on 7 x 143gb drives. I did make clone of my zfs file systems with their snaps and something is not right, sizes do not match... anyway here is what I have: [17:50:32] [EMAIL PROTECTED]: /root > zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 272G 1.95G 24.5K /mypool mypool/d 272G 1.95G 143G /d/d2 mypool/[EMAIL PROTECTED] 3.72G - 123G - mypool/[EMAIL PROTECTED] 22.3G - 156G - mypool/[EMAIL PROTECTED] 23.3G - 161G - mypool/[EMAIL PROTECTED] 16.1G - 172G - mypool/[EMAIL PROTECTED] 13.8G - 168G - mypool/[EMAIL PROTECTED] 15.7G - 168G - mypool/[EMAIL PROTECTED]192M - 143G - mypool/[EMAIL PROTECTED]189M - 143G - mypool/[EMAIL PROTECTED]200M - 143G - mypool/[EMAIL PROTECTED] 3.93M - 143G - mypool2463G 474G52K /mypool2 mypool2/d 318G 474G 168G /mypool2/d mypool2/[EMAIL PROTECTED] 4.40G - 145G - mypool2/[EMAIL PROTECTED] 26.1G - 184G - mypool2/[EMAIL PROTECTED] 27.3G - 189G - mypool2/[EMAIL PROTECTED] 18.7G - 202G - mypool2/[EMAIL PROTECTED] 16.1G - 197G - mypool2/[EMAIL PROTECTED]18.2G - 198G - mypool2/d3 145G 474G 145G legacy see: mypool/d 272G 1.95G 143G /d/d2 mypool2/d 318G 474G 168G /mypool2/d they are the same copies but their sizes do differ quite a bit, original is 272G and the "clone" which I duplicated by zfs send/receive is 318G. Then all the other snaps also do differ. Why is that difference? Could someone explain how does it work? Regards, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
