On 12/23/12 12:28 PM, Jason Evans wrote:
On Dec 21, 2012, at 7:37 PM, Alfred Perlstein <bri...@mu.org> wrote:
So the other day in an effort to debug a memory leak I decided to take a look
at malloc+utrace(2) and decided to make a tool to debug where leaks are coming
from.
A few hours later I have:
1) a new version of utrace(2) (utrace2(2)) that uses structured data to prevent
overloading of data. (utrace2.diff)
2) changes to ktrace and kdump to decode the new format. (also in utrace2.diff)
3) changes to jemalloc to include the new format AND the function caller so
it's easy to get the source of the leaks. (also in utrace2.diff)
4) a program that can take a pipe of kdump(1) and figure out what memory has
leaked. (alloctrace.py)
5) simple test program (test_utrace.c)
[…]
Have you looked at the heap profiling functionality built into jemalloc? It's not
currently enabled on FreeBSD, but as far as I know, the only issue keeping it from
being useful is the absence of a Linux-compatible /proc/<pid>/maps (and the
gperftools folks may already have a solution for that; I haven't looked). I think it
makes more sense to get that sorted out than to develop a separate trace-based leak
checker. The problem with tracing is that it doesn't scale beyond some relatively
small number of allocator events.
I have looked at some of this functionality (heap profiling) but alas it
is not implemented yet. In addition the dtrace work appears to be quite
away from a workable solution with too many performance penalties until
some serious hacking is done.
I am just not sure how to proceed, on one hand I do not really have the
skill to fix the /proc/pid/maps problem, nor figure out how to get
dtrace into the system in any time frame that is reasonable.
All a few of us need is the addition of the trace back into the existing
utrace framework.
Is it time to start installing with some form of debug symbols? This would help
us also with dtrace.
Re: debug symbols, frame pointers, etc. necessary to make userland dtrace work
by default, IMO we should strongly prefer such defaults. It's more reasonable
to expect people who need every last bit of performance to remove functionality
than to expect people who want to figure out what the system is doing to figure
out what functionality to turn on.
This is very true. I'm going to continue to work towards this end with
a few people and get up to speed on it so that hopefully we can get to
this point hopefully in the next release cycle or two.
If you have a few moments, can you have a look at the "utrace2" branches
here:
https://github.com/alfredperlstein/freebsd/tree/utrace2
This branch contains the addition of the utrace2 system call which is
needed to structure data via utrace(2). The point of this is to avoid
kdump(1) needing to discern type of ktrace records based on arbitrary
size or other parameters and introduces an extensible protocol for new
types of utrace data.
The utrace2 branch here augments jemalloc to use utrace2 to pass the old
utrace records, but in addition to pass the return address along with
the type and size of the allocation:
https://github.com/alfredperlstein/jemalloc/tree/utrace2
-Alfred
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
