Re: ZFS Tuning - arc_summary.pl
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Mon, 29 Mar 2010 10:43, Barry Pederson wrote:
In Message-Id: <4bb0bc7c.3000...@barryp.org>
I've been using the arc_summary.pl script from here:
http://jhell.googlecode.com/svn/base/head/scripts/zfs/arc_summary/arc_summary.pl
and noticed some odd numbers, with the ARC Current Size being larger than the
Max Size, and the breakdown adding up to less than the current size as shown
below
ARC Size:
Current Size: 992.71M (arcsize)
Target Size: (Adaptive) 512.00M (c)
Min Size (Hard Limit): 81.82M (arc_min)
Max Size (Hard Limit): 512.00M (arc_max)
ARC Size Breakdown:
Recently Used Cache Size: 99.84% 511.18M (p)
Frequently Used Cache Size: 0.16% 0.82M (c-p)
From another thread I saw, it sounds like arc_max isn't really
a "Hard Limit" but rather some kind of high water mark. If that's
the case then I wonder if this might make more sense
-
--- arc_summary.pl.original 2010-02-25 19:23:13.0 -0600
+++ arc_summary.pl 2010-03-29 09:32:28.0 -0500
@@ -121,20 +121,20 @@
my $arc_size = ${Kstat}->{zfs}->{0}->{arcstats}->{size};
my $arc_size_MiB = ($arc_size / 1048576);
-my $mfu_size = $target_size - $mru_size;
+my $mfu_size = $arc_size - $mru_size;
my $mfu_size_MiB = ($mfu_size / 1048576);
-my $mru_perc = 100*($mru_size / $target_size);
-my $mfu_perc = 100*($mfu_size / $target_size);
+my $mru_perc = 100*($mru_size / $arc_size);
+my $mfu_perc = 100*($mfu_size / $arc_size);
print "ARC Size:\n";
printf("\tCurrent Size:\t\t\t\t%0.2fM (arcsize)\n", $arc_size_MiB);
printf("\tTarget Size: (Adaptive)\t\t\t%0.2fM (c)\n", $target_size_MiB);
printf("\tMin Size (Hard Limit):\t\t\t%0.2fM (arc_min)\n",
$arc_min_size_MiB);
-printf("\tMax Size (Hard Limit):\t\t\t%0.2fM (arc_max)\n",
$arc_max_size_MiB);
+printf("\tMax Size :\t\t\t%0.2fM (arc_max)\n",
$arc_max_size_MiB);
print "\nARC Size Breakdown:\n";
printf("\tRecently Used Cache Size:\t%0.2f%%\t%0.2fM (p)\n", $mru_perc,
$mru_size_MiB);
-printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (c-p)\n", $mfu_perc,
$mfu_size_MiB);
+printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (arcsize-p)\n",
$mfu_perc, $mfu_size_MiB);
print "\n";
### ARC Efficency ###
---
Giving something like this...
ARC Size:
Current Size: 992.88M (arcsize)
Target Size: (Adaptive) 512.00M (c)
Min Size (Hard Limit): 81.82M (arc_min)
Max Size : 512.00M (arc_max)
ARC Size Breakdown:
Recently Used Cache Size: 51.48% 511.18M (p)
Frequently Used Cache Size: 48.52% 481.70M (arcsize-p)
Barry
Barry,
What branch and revision was this run on ?
I need the above information because the output above just does not match
up quite as it should and I want to investigate when, where & why as I
believe something else is going on here that is not on the behalf of
arc_summary.pl.
Or if you could provide me personally with the full output of the script
from the downloads section "just to be sure" in an attachment that would
work as well.
Thanks.
- --
jhell
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.14 (FreeBSD)
iQEcBAEBAgAGBQJLzUAhAAoJEJBXh4mJ2FR+kP4H/3FSkazC+Jxv0q5XJhP/YfeP
gJ0vWP+84J6HM68GyS4eCOu3QPGUPBAuqZOS8Bb9jXg9xNfxCvw2DQn5mP6v6i6H
w8mWyYyCla7iBfItod4L2GjQeP52SIt7sW9icDeWvrS+LphwjTQmBiA4QwGBQT5D
YiarUMpzY1Jkq8I6YgGYRIwZqeuNn7X68ZEKIz8/LhTM6WKdktm5dcBb6UM/mC/a
I82sv+7mG/9Bn0Orp7DMqvym0rllYmb+Sj7Pj2NEcPt9LYDNf6Vy1Wmly6hNQTYb
b8WkfgLeMogDN9JS6Bw+UxNGwHgQgqDIWvkKDt9qrmuTpKLEozD6GnBzo27uZkg=
=RoRd
-END PGP SIGNATURE-
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS Tuning - arc_summary.pl
On 03/29/2010 12:11, Jeremy Chadwick wrote: > On Mon, Mar 29, 2010 at 09:43:08AM -0500, Barry Pederson wrote: >> From another thread I saw, it sounds like arc_max isn't really >> a "Hard Limit" but rather some kind of high water mark. If that's >> the case then I wonder if this might make more sense > > It became a hard limit in a semi-recent commit somewhere. I've lost > count of the modifications at this point. So, the perl script would > have to read __FreeBSD_version in /usr/include/osreldate.h and adjust > its output accordingly. > Update: The current version in head r52 has these modifications now. As a side note while doing the the original modification I noticed some strange values (negative) and decided to just branch what I already had in head to stable/7 and continue with the modifications in head. arc_summary.pl has not been updated in the downloads section until I investigate more what happened with those negative values and why. I will probably be adding some more (2) lines to the ARC Size Breakdown area that compares its current usage to the the target_size so a comparison can be done side-by-side of the difference to current size and what it would look like at the target_size. Regards, -- jhell ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS Tuning - arc_summary.pl
On 3/30/10 9:30 AM, jhell wrote: Ill mark this up on my todo. Thanks for the feedback& I should have something committed back within the next couple days, possibly even tonight. I never recalculated for this difference as that area of the code was just a formatting fix. But as Jeremy has pointed out, it would have to verify against __FreeBSD_version but since I already pull in the sysctl MIB kern.osreldate I should be able to compare to that and say 70 or higher make the above correction. As this is mainly ZFS v13 dependent now I don't feel to bad doing what I have stated above. Thanks again, Regards, Shouldn't the "ARC Size Breakdown" be based on the current size (instead of the target size) no matter what? If the current size was *less* than arc_max, you'd also get an nonsensical breakdown - in that case with the "recently used" and "frequently used" adding up to *more* than the current size. It seems to me that the only thing that would be different between newer and older FreeBSD version would be whether arc_max would be described as a "Hard Limit" or not - which is maybe not that important to show. If that's all true, then the patch should work for both older and newer versions of FreeBSD. Barry ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS Tuning - arc_summary.pl
On 03/29/2010 10:43, Barry Pederson wrote:
> I've been using the arc_summary.pl script from here:
>
> http://jhell.googlecode.com/svn/base/head/scripts/zfs/arc_summary/arc_summary.pl
>
>
> and noticed some odd numbers, with the ARC Current Size being larger
> than the Max Size, and the breakdown adding up to less than the current
> size as shown below
>
>
> ARC Size:
> Current Size: 992.71M (arcsize)
> Target Size: (Adaptive) 512.00M (c)
> Min Size (Hard Limit): 81.82M (arc_min)
> Max Size (Hard Limit): 512.00M (arc_max)
>
> ARC Size Breakdown:
> Recently Used Cache Size: 99.84% 511.18M (p)
> Frequently Used Cache Size: 0.16% 0.82M (c-p)
>
>
>
> From another thread I saw, it sounds like arc_max isn't really
> a "Hard Limit" but rather some kind of high water mark. If that's
> the case then I wonder if this might make more sense
>
>
>
> -
> --- arc_summary.pl.original 2010-02-25 19:23:13.0 -0600
> +++ arc_summary.pl 2010-03-29 09:32:28.0 -0500
> @@ -121,20 +121,20 @@
>
> my $arc_size = ${Kstat}->{zfs}->{0}->{arcstats}->{size};
> my $arc_size_MiB = ($arc_size / 1048576);
> -my $mfu_size = $target_size - $mru_size;
> +my $mfu_size = $arc_size - $mru_size;
> my $mfu_size_MiB = ($mfu_size / 1048576);
> -my $mru_perc = 100*($mru_size / $target_size);
> -my $mfu_perc = 100*($mfu_size / $target_size);
> +my $mru_perc = 100*($mru_size / $arc_size);
> +my $mfu_perc = 100*($mfu_size / $arc_size);
>
> print "ARC Size:\n";
> printf("\tCurrent Size:\t\t\t\t%0.2fM (arcsize)\n", $arc_size_MiB);
> printf("\tTarget Size: (Adaptive)\t\t\t%0.2fM (c)\n", $target_size_MiB);
> printf("\tMin Size (Hard Limit):\t\t\t%0.2fM (arc_min)\n",
> $arc_min_size_MiB);
> -printf("\tMax Size (Hard Limit):\t\t\t%0.2fM (arc_max)\n",
> $arc_max_size_MiB);
> +printf("\tMax Size :\t\t\t%0.2fM (arc_max)\n",
> $arc_max_size_MiB);
>
> print "\nARC Size Breakdown:\n";
> printf("\tRecently Used Cache Size:\t%0.2f%%\t%0.2fM (p)\n", $mru_perc,
> $mru_size_MiB);
> -printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (c-p)\n",
> $mfu_perc, $mfu_size_MiB);
> +printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (arcsize-p)\n",
> $mfu_perc, $mfu_size_MiB);
> print "\n";
>
> ### ARC Efficency ###
>
> ---
>
>
> Giving something like this...
>
>
> ARC Size:
> Current Size: 992.88M (arcsize)
> Target Size: (Adaptive) 512.00M (c)
> Min Size (Hard Limit): 81.82M (arc_min)
> Max Size : 512.00M (arc_max)
>
> ARC Size Breakdown:
> Recently Used Cache Size: 51.48% 511.18M (p)
> Frequently Used Cache Size: 48.52% 481.70M (arcsize-p)
>
>
> Barry
> ___
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Ill mark this up on my todo. Thanks for the feedback & I should have
something committed back within the next couple days, possibly even tonight.
I never recalculated for this difference as that area of the code was
just a formatting fix.
But as Jeremy has pointed out, it would have to verify against
__FreeBSD_version but since I already pull in the sysctl MIB
kern.osreldate I should be able to compare to that and say 70 or
higher make the above correction.
As this is mainly ZFS v13 dependent now I don't feel to bad doing what I
have stated above.
Thanks again,
Regards,
--
jhell
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS Tuning - arc_summary.pl
On Mar 29, 2010, at 10:43 AM, Barry Pederson wrote: > I've been using the arc_summary.pl script from here: > > http://jhell.googlecode.com/svn/base/head/scripts/zfs/arc_summary/arc_summary.pl > > and noticed some odd numbers, with the ARC Current Size being larger than the > Max Size, and the breakdown adding up to less than the current size as shown > below I believe the current size can be larger than the max if your working data set is large enough. The ARC can't evict data that is still being referenced. When I last looked (over a year ago) there was no mechanism to provide back pressure from the ARC to the VM layer to request that more nodes be released to help deal with this situation. I don't know if that really helps at all, but I just thought I would add the data point from my previous debugging sessions with zfs. - Ben___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS Tuning - arc_summary.pl
On Mon, Mar 29, 2010 at 09:43:08AM -0500, Barry Pederson wrote: > From another thread I saw, it sounds like arc_max isn't really > a "Hard Limit" but rather some kind of high water mark. If that's > the case then I wonder if this might make more sense It became a hard limit in a semi-recent commit somewhere. I've lost count of the modifications at this point. So, the perl script would have to read __FreeBSD_version in /usr/include/osreldate.h and adjust its output accordingly. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
ZFS Tuning - arc_summary.pl
I've been using the arc_summary.pl script from here:
http://jhell.googlecode.com/svn/base/head/scripts/zfs/arc_summary/arc_summary.pl
and noticed some odd numbers, with the ARC Current Size being larger than the Max Size, and the breakdown adding up to
less than the current size as shown below
ARC Size:
Current Size: 992.71M (arcsize)
Target Size: (Adaptive) 512.00M (c)
Min Size (Hard Limit): 81.82M (arc_min)
Max Size (Hard Limit): 512.00M (arc_max)
ARC Size Breakdown:
Recently Used Cache Size: 99.84% 511.18M (p)
Frequently Used Cache Size: 0.16% 0.82M (c-p)
From another thread I saw, it sounds like arc_max isn't really
a "Hard Limit" but rather some kind of high water mark. If that's
the case then I wonder if this might make more sense
-
--- arc_summary.pl.original 2010-02-25 19:23:13.0 -0600
+++ arc_summary.pl 2010-03-29 09:32:28.0 -0500
@@ -121,20 +121,20 @@
my $arc_size = ${Kstat}->{zfs}->{0}->{arcstats}->{size};
my $arc_size_MiB = ($arc_size / 1048576);
-my $mfu_size = $target_size - $mru_size;
+my $mfu_size = $arc_size - $mru_size;
my $mfu_size_MiB = ($mfu_size / 1048576);
-my $mru_perc = 100*($mru_size / $target_size);
-my $mfu_perc = 100*($mfu_size / $target_size);
+my $mru_perc = 100*($mru_size / $arc_size);
+my $mfu_perc = 100*($mfu_size / $arc_size);
print "ARC Size:\n";
printf("\tCurrent Size:\t\t\t\t%0.2fM (arcsize)\n", $arc_size_MiB);
printf("\tTarget Size: (Adaptive)\t\t\t%0.2fM (c)\n", $target_size_MiB);
printf("\tMin Size (Hard Limit):\t\t\t%0.2fM (arc_min)\n", $arc_min_size_MiB);
-printf("\tMax Size (Hard Limit):\t\t\t%0.2fM (arc_max)\n", $arc_max_size_MiB);
+printf("\tMax Size :\t\t\t%0.2fM (arc_max)\n", $arc_max_size_MiB);
print "\nARC Size Breakdown:\n";
printf("\tRecently Used Cache Size:\t%0.2f%%\t%0.2fM (p)\n", $mru_perc,
$mru_size_MiB);
-printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (c-p)\n", $mfu_perc,
$mfu_size_MiB);
+printf("\tFrequently Used Cache Size:\t%0.2f%%\t%0.2fM (arcsize-p)\n",
$mfu_perc, $mfu_size_MiB);
print "\n";
### ARC Efficency ###
---
Giving something like this...
ARC Size:
Current Size: 992.88M (arcsize)
Target Size: (Adaptive) 512.00M (c)
Min Size (Hard Limit): 81.82M (arc_min)
Max Size : 512.00M (arc_max)
ARC Size Breakdown:
Recently Used Cache Size: 51.48% 511.18M (p)
Frequently Used Cache Size: 48.52% 481.70M (arcsize-p)
Barry
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
