Here's the top output from this morning: Mem: 1300M Active, 292M Inact, 295M Wired, 104M Cache, 112M Buf, 5116K Free Swap: 1024M Total, 8036K Used, 1016M Free
At this time there are ~130 processes, whereas in the previous one there were ~76, with more memory marked as "Active". vmstat says this: procs memory page disks faults cpu r b w avm fre flt re pi po fr sr da0 ac0 in sy cs us sy id 0 0 1 848048 100880 937 113 23 0 1503 248 0 0 2678 0 8211 2 4 95 0 0 1 830192 110484 340 4 1 0 2719 0 1 0 2574 0 5917 1 5 93 avm is supposed to be the number of active virtual pages. The page size being 4096, according to hw.pagesize, would actually come up to around 3.4GB. I'm not sure what this number refers to - maybe it's memory + swap? The whole reason I'm bringing this up with the list is that, ultimately, this is causing the machine to crash, after running out of swap. If I could identify where this Active memory has gone, I may be able to recover the "lost" memory and prevent the server from swapping itself to death. Here's the ps -ax output - note that nfsd is running, but it's not being used. It does have an NFS client mounted, but from "vmstat -m", it isn't using any memory at all (and it would be in Buf I think?) PID TT STAT TIME COMMAND 0 ?? DLs 0:00.82 (swapper) 0 ?? ZW 0:00.00 (httpd) 1 ?? ILs 0:00.08 /sbin/init -- 2 ?? DL 0:13.59 (g_event) 3 ?? DL 3:41.95 (g_up) 4 ?? DL 4:56.15 (g_down) 5 ?? DL 1:41.88 (pagedaemon) 6 ?? DL 0:00.00 (vmdaemon) 7 ?? DL 16:25.54 (pagezero) 8 ?? DL 2:07.11 (bufdaemon) 9 ?? DL 0:01.51 (vnlru) 10 ?? DL 0:00.00 (ktrace) 11 ?? RL 0:00.00 (idle: cpu3) 12 ?? RL 5037:43.41 (idle: cpu2) 13 ?? RL 0:00.00 (idle: cpu1) 14 ?? RL 4281:37.99 (idle: cpu0) 15 ?? WL 127:48.20 (swi1: net) 16 ?? WL 6:02.49 (swi7: tty:sio clock) 18 ?? DL 1:26.21 (random) 20 ?? WL 1:39.12 (swi3: cambio) 23 ?? WL 0:00.00 (irq14: ata0) 26 ?? DL 0:00.02 (usb0) 27 ?? DL 0:00.00 (usbtask) 30 ?? WL 2:13.96 (irq16: ahc0) 31 ?? WL 0:00.00 (irq17: ahc1) 32 ?? WL 58:17.09 (irq18: em0) 33 ?? WL 0:00.00 (irq1: atkbd0) 34 ?? WL 0:00.00 (irq6: fdc0) 35 ?? WL 0:00.00 (swi0: tty:sio) 40 ?? DL 13:22.17 (syncer) 41 ?? IL 3:06.26 (nfsiod 0) 42 ?? IL 2:47.50 (nfsiod 1) 43 ?? IL 2:28.19 (nfsiod 2) 44 ?? IL 3:37.52 (nfsiod 3) 148 ?? Is 0:00.00 adjkerntz -i 257 ?? Ss 0:02.08 /usr/sbin/syslogd -s 352 ?? Ss 0:00.25 /usr/sbin/rpcbind 375 ?? Is 0:00.01 /usr/sbin/mountd -r 377 ?? Is 0:00.02 nfsd: master (nfsd) 380 ?? I 0:00.00 nfsd: server (nfsd) 381 ?? I 0:00.00 nfsd: server (nfsd) 382 ?? I 0:00.00 nfsd: server (nfsd) 383 ?? I 0:00.00 nfsd: server (nfsd) 394 ?? Ss 0:00.35 /usr/sbin/usbd 443 ?? Is 0:02.00 /usr/sbin/sshd -u0 449 ?? Ss 0:06.17 sendmail: accepting connections (sendmail) 452 ?? Is 0:00.13 sendmail: Queue [EMAIL PROTECTED]:30:00 for /var/spool/client 466 ?? Is 0:01.25 /usr/sbin/cron 829 ?? Ss 0:02.75 proftpd: (accepting connections) (proftpd) 855 ?? Is 0:00.00 /usr/sbin/inetd -wW 21727 ?? Ss 2:11.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 31784 ?? S 0:00.23 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 52298 ?? S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 52989 ?? S 0:00.18 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 53224 ?? S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 54180 ?? S 0:00.18 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 54282 ?? S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 54310 ?? S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 55563 ?? S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 55970 ?? S 0:00.16 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56261 ?? S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56369 ?? S 0:00.10 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56422 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56424 ?? S 0:00.18 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56465 ?? S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56491 ?? S 0:00.17 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56523 ?? S 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56575 ?? Rs 0:00.07 sshd: [EMAIL PROTECTED] (sshd) 56606 ?? S 0:00.33 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56617 ?? S 0:00.15 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56729 ?? I 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56763 ?? S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56768 ?? S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56779 ?? S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56786 ?? S 0:00.05 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56788 ?? S 0:00.00 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56789 ?? S 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56790 ?? S 0:00.08 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56800 ?? S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56809 ?? I 0:00.01 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56811 ?? S 0:00.13 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56840 ?? S 0:00.15 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56843 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56845 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56846 ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56852 ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56853 ?? S 0:00.05 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56864 ?? S 0:00.11 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56865 ?? S 0:00.17 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56866 ?? S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56869 ?? S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56870 ?? S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56871 ?? S 0:00.05 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56872 ?? S 0:00.09 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56873 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56874 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56875 ?? S 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56876 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56877 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56878 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56879 ?? S 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56880 ?? S 0:00.07 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56881 ?? S 0:00.06 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56882 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56883 ?? S 0:00.05 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56884 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56885 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56886 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56887 ?? S 0:00.12 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56888 ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56889 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56890 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56891 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56892 ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56893 ?? S 0:00.04 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56894 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56896 ?? S 0:00.00 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56897 ?? S 0:00.02 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56898 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56900 ?? S 0:00.03 /usr/local/bin/httpd -f /usr/local/www/server/conf/ht 56577 p0 Ss 0:00.02 -sh (sh) 56904 p0 R+ 0:00.00 ps -ax 3574 v0 Is+ 0:00.00 /usr/libexec/getty Pc ttyv0 868 v1 Is+ 0:00.00 /usr/libexec/getty Pc ttyv1 869 v2 Is+ 0:00.00 /usr/libexec/getty Pc ttyv2 870 v3 Is+ 0:00.00 /usr/libexec/getty Pc ttyv3 871 v4 Is+ 0:00.00 /usr/libexec/getty Pc ttyv4 872 v5 Is+ 0:00.00 /usr/libexec/getty Pc ttyv5 873 v6 Is+ 0:00.00 /usr/libexec/getty Pc ttyv6 874 v7 Is+ 0:00.00 /usr/libexec/getty Pc ttyv7 520 con- I 0:02.81 /usr/local/sbin/snmpd 534 con- I 0:00.01 /bin/sh /usr/local/bin/mysqld_safe --datadir=/usr/loc 566 con- S 138:26.09 /usr/local/libexec/mysqld --basedir=/usr/local --data According to top and ps, mysql is only using around 59M VSZ. On Sun, 1 Feb 2004, Jorn Argelo wrote: > Are you sure that there isn't anything else running? Why don't you give us > an ps -ax output? I don't think there's a memory leak in 5.1, since I've > seen running 5.1 just fine on a PE2650 with 2 GB RAM. You shouldn't > rely on top too much acually. Vmstat is a better program when looking > at memory. > > Cheers, > > Jorn > ----- Original Message ----- > From: "dpk" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Saturday, January 31, 2004 1:10 AM > Subject: Memory leak on 5.1-RELEASE? > > > > (I'm not a member of the list; please Cc me on any replies.) > > > > We're running Apache 1.3.28 on a 5.1-RELEASE machine. It's a Dell PE 2650 > > w/ 2GB RAM. The site contains a lot of large files (multi-megabyte) - > > otherwise there's nothing unusual running. > > > > The Active memory use, according to top, seems rather high: > > > > last pid: 21487; load averages: 0.19, 0.33, 0.32 up 2+16:45:20 > 15:52:21 > > 76 processes: 1 running, 75 sleeping > > CPU states: 0.5% user, 0.0% nice, 4.0% system, 1.4% interrupt, 94.2% > idle > > Mem: 1413M Active, 187M Inact, 299M Wired, 93M Cache, 112M Buf, 2632K Free > > Swap: 1024M Total, 21M Used, 1003M Free, 2% Inuse > > > > We can't seem to get the Active number down much, even after stopping > > Apache it still stays around 1100M. There's no shared memory in use, and > > nothing in vmstat -m seems to indicate where the "missing" memory is. top, > > sorting by "size", does not indicate anything unusual either. > > > > sysctl vm.vmtotal says: > > > > vm.vmtotal: > > System wide totals computed every five seconds: (values in kilobytes) > > =============================================== > > Processes: (RUNQ: 1 Disk Wait: 0 Page Wait: 0 Sleep: 76) > > Virtual Memory: (Total: 8172K, Active 636472K) > > Real Memory: (Total: 2051312K Active 389176K) > > Shared Virtual Memory: (Total: 16436K Active: 11760K) > > Shared Real Memory: (Total: 6004K Active: 4436K) > > Free Memory Pages: 79228K > > > > whereas on other servers, the Real Memory "Active" number seems to match > > the one found in top, on this one it is about 1GB lower. > > > > A similar machine running Apache on 5.1-R, generally serving smaller > > files, has the same problem in a smaller scale (about 640M even when > > Apache is stopped). > > > > Are there any other data that I should send to help diagnose this problem, > > or any programs I can run to try and track this stray memory use down? > > > > - dpk > > _______________________________________________ > > [EMAIL PROTECTED] mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > > To unsubscribe, send any mail to > "[EMAIL PROTECTED]" > > > > _______________________________________________ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to "[EMAIL PROTECTED]"