Hey,
We are using heavily sysperfstat, to monitor CPU, Mem utilisation/saturation on
many systems
running Solaris 10, SPARC and X86.
On some systems sysperfstat started to return negative
values for memory utilisation:
------ Utilisation ------ ------ Saturation ------
Time %CPU %Mem %Disk %Net CPU Mem Disk Net
10:38:23 9.78 -3.06 5.13 3.50 0.19 0.00 0.00 0.00
10:38:24 8.33 -3.06 3.35 0.14 0.00 0.00 0.00 0.00
10:38:25 8.15 -3.06 0.00 0.13 0.00 0.00 0.00 0.00
10:38:26 8.74 -3.06 0.00 0.17 0.00 0.00 0.00 0.00
10:38:27 8.72 -3.06 6.32 0.34 0.00 0.00 0.00 0.00
10:38:28 8.28 -3.06 0.00 0.07 0.00 0.00 0.00 0.00
10:38:30 8.34 -3.06 6.99 0.18 0.00 0.00 0.00 0.00
10:38:31 9.35 -3.06 4.38 0.18 0.00 0.00 0.00 0.00
10:38:32 8.59 -3.06 11.13 0.13 0.00 0.00 0.00 0.00
10:38:33 8.33 -3.06 0.00 0.09 0.00 0.00 0.00 0.00
Average 8.66 -3.06 3.73 0.49
This comes because availrmem < freemem. Checking sysperfstat:
# Process utilisation.
# this is a little odd, most values from kstat are incremental
# however these are absolute. we calculate and return the final
# value as a percentage. page conversion is not necessary as
# we divide that value away.
#
$pct = 100 - 100 * ($freemem / $availrmem);
This is easily visible from kstat:
module: unix instance: 0
name: system_pages class: pages
availrmem 1864890
crtime 276.453526475
desfree 32132
desscan 25
econtig 50331648
fastscan 160637
freemem 1921991
kernelbase 16777216
lotsfree 64265
minfree 16066
nalloc 30194443
nalloc_calls 12891
nfree 27592693
nfree_calls 8788
nscan 0
pagesfree 1921991
pageslocked 2248092
pagestotal 4112982
physmem 4179383
pp_kernel 288436
slowscan 100
snaptime 13561898.6211937
This systems runs 4 zones hosting 20 JVMs. The memory consumers
/utilisation via mdb below:
> ::memstat
Page Summary Pages MB %Tot
------------ ---------------- ---------------- ----
Kernel 209750 1638 5%
Anon 1701252 13291 41%
Exec and libs 12214 95 0%
Page cache 269012 2101 6%
Free (cachelist) 1252352 9784 30%
Free (freelist) 734803 5740 18%
Total 4179383 32651
Physical 4112982 3213
A lot of Anon allocations, most likely caused by the big number of JVMs.
Questions:
1). In what conditions can availrmem fall below freemem ? Availrmem
does describe the resident,unswappable pages from system and it is not static,
right ?
What could trigger availrmem < freemem - Kernel allocations, bugs ...
2). Is this a normal situation ?
3). How one would interpret the memory utilisation in this case ?
Im trying to patch sysperfstat.
Rgds,
stefan
PS: I have noticed negative values on majority of our app servers, which run
lots of JVMs.
Example:
System Configuration: Sun Microsystems sun4v Sun Fire T200
System clock frequency: 200 MHz
Memory size: 32760 Megabytes
Solaris 10 11/06 s10s_u3wos_10 SPARC
Copyright 2006 Sun Microsystems, Inc. All Rights Reserved.
Use is subject to license terms.
Assembled 14 November 2006
--
This messages posted from opensolaris.org
