Re: [zfs-discuss] ZFS snapshot used space question

2012-08-31 Thread Truhn, Chad


Is there a way to get the total amount of data referenced by a snapshot that 
isn't referenced by a specified snapshot/filesystem?  I think this is what is 
really desired in order to locate snapshots with offending space usage.  The 
written and written@ attributes seem to only do the reverse.  I think you can 
back calculate it from the snapshot and filesystem referenced sizes, and 
the written@snap property of the filesystem, but that isn't particularly 
convenient to do (looks like zfs get -Hp ... makes it possible to hack a 
script together for, though).


This is what I was hoping to get as well, but I am not sure it's really 
possible.  Even if you try to calculate the referenced space + displayed used 
space and compare against the active filesystem that doesn't really tell you 
much because the data on the active filesystem might not be as static as you 
want.  

For example:

If it references 10G and the active filesystem shows 10G used, you might expect 
that the snapshot isn't using any space.  However, the 10G it referenced might 
have been deleted and the 10G in the active filesystem might be new data and 
that means your snap could be 10G.  But if 9G of that was on another snapshot, 
you would have something like this:

rootpool/export/home@snap.0-   1G-   -  -   
   -
rootpool/export/home@snap.1-   27K  -   -  
-  -
rootpool/export/home@snap.2-   0   -   -  - 
 -

And the referenced would look something like:

rootpool/export/home@snap.00  -  10G  -
rootpool/export/home@snap.10  -  9G-
rootpool/export/home@snap.10  -  10G-

And the current filesystem would be:

rootpool/export/home  40G  20G 10G   10G
  0  0


Then imagine that across more than three snapshots.  I can't wrap my head 
around logic that would work there.

I would love if someone could figure out a good way though...

- Chad




___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss


Re: [zfs-discuss] ZFS snapshot used space question

2012-08-31 Thread Timothy Coalson
I went ahead and hacked this script together, so let me elaborate.  First,
though, a teaser:

$ ./snapspace.sh mainpool/storage
SNAPSHOTOLDREFS UNIQUE  UNIQUE%
zfs-auto-snap_monthly-2011-11-14-18h59  34.67G  11.0G   31%
zfs-auto-snap_monthly-2011-12-14-18h59  33.41G  8.95G   26%
zfs-auto-snap_monthly-2012-02-09-16h05  123.75G 70.3G   56%
...

OLDREFS is how much of the referenced space of the snapshot is not
referenced by the current filesystem.  UNIQUE is the amount that is
referenced by that snapshot and nothing else (the used property), and
finally I thought it would be nice to have the unique amount as a
percentage of OLDREFS.

So, the heart of the script is this:

...
fullref=`zfs get -Hp referenced $1 | awk '{print $3}'`
for snap in `zfs list -Hd 1 -t snapshot -o name $1 | cut -f2- -d@`
do
snapref=`zfs get -Hp referenced $1@$snap | awk '{print $3}'`
snapwritten=`zfs get -Hp written@$snap $1 | awk '{print $3}'`
 olddata=$((snapref + snapwritten - fullref))
...

Everything else is sanity checking and display formatting.  So, it adds the
snapshot referenced data, and the written@snap value of the filesystem
(Richard, thanks again for pointing this out), which would be the size of
the current filesystem if nothing was deleted/overwritten (which are
equivalent due to copy on write), and subtracts the actual size of the
current filesystem, obtaining the amount of data deleted/overwritten since
the snapshot was taken (or to put it another way, all the data in that
snapshot that isn't in the current filesystem).

Since this relies on the written@ property, I think it will only work on
illumos-based distros, based on what Richard said (apologies to everyone
that doesn't use an illumos kernel, if so).  Also, since the values are
retrieved with separate commands, if the filesystem is being written to,
the numbers may become strange, but probably only for the most recent
snapshots.  Attaching the full script (renamed to .txt because the listserv
hates scripts) for all who are interested.

Tim

On Fri, Aug 31, 2012 at 8:04 AM, Truhn, Chad
chad.tr...@bowheadsupport.comwrote:



 Is there a way to get the total amount of data referenced by a snapshot
 that isn't referenced by a specified snapshot/filesystem?  I think this is
 what is really desired in order to locate snapshots with offending space
 usage.  The written and written@ attributes seem to only do the reverse.
  I think you can back calculate it from the snapshot and filesystem
 referenced sizes, and the written@snap property of the filesystem,
 but that isn't particularly convenient to do (looks like zfs get -Hp ...
 makes it possible to hack a script together for, though).


 This is what I was hoping to get as well, but I am not sure it's really
 possible.  Even if you try to calculate the referenced space + displayed
 used space and compare against the active filesystem that doesn't really
 tell you much because the data on the active filesystem might not be as
 static as you want.

 For example:

 If it references 10G and the active filesystem shows 10G used, you might
 expect that the snapshot isn't using any space.  However, the 10G it
 referenced might have been deleted and the 10G in the active filesystem
 might be new data and that means your snap could be 10G.  But if 9G of that
 was on another snapshot, you would have something like this:

 rootpool/export/home@snap.0-   1G-   -
-  -
 rootpool/export/home@snap.1-   27K  -   -
  -  -
 rootpool/export/home@snap.2-   0   -   -
  -  -

 And the referenced would look something like:

 rootpool/export/home@snap.00  -  10G  -
 rootpool/export/home@snap.10  -  9G-
 rootpool/export/home@snap.10  -  10G-

 And the current filesystem would be:

 rootpool/export/home  40G  20G 10G   10G
0  0


 Then imagine that across more than three snapshots.  I can't wrap my head
 around logic that would work there.

 I would love if someone could figure out a good way though...

 - Chad





#!/bin/bash
if (($#  1))
then
echo usage: $0 filesystem
exit 1
fi
if [[ $1 == *@* || $1 == /* ]]
then
echo Snapshots and paths are not supported
echo usage: $0 filesystem
exit 1
fi
echo SNAPSHOTOLDREFS   \tUNIQUE\tUNIQUE%
fullref=`zfs get -Hp referenced $1 | awk '{print $3}'`
for snap in `zfs list -Hd 1 -t snapshot -o name $1 | cut -f2- -d@`
do
snapref=`zfs get -Hp referenced $1@$snap | awk '{print $3}'`
snapwritten=`zfs get -Hp written@$snap $1 | awk '{print $3}'`
olddata=$((snapref + snapwritten - fullref))
snapused=`zfs get -H used $1@$snap | awk '{print $3}'`
snapusedraw=`zfs get -Hp used $1@$snap | awk '{print $3}'`