On 12/23/12 9:28 AM, Jason Evans wrote:
On Dec 21, 2012, at 7:37 PM, Alfred Perlstein <bri...@mu.org> wrote:
So the other day in an effort to debug a memory leak I decided to take a look
at malloc+utrace(2) and decided to make a tool to debug where leaks are coming
from.
A few hours later I have:
1) a new version of utrace(2) (utrace2(2)) that uses structured data to prevent
overloading of data. (utrace2.diff)
2) changes to ktrace and kdump to decode the new format. (also in utrace2.diff)
3) changes to jemalloc to include the new format AND the function caller so
it's easy to get the source of the leaks. (also in utrace2.diff)
4) a program that can take a pipe of kdump(1) and figure out what memory has
leaked. (alloctrace.py)
5) simple test program (test_utrace.c)
[…]
Have you looked at the heap profiling functionality built into jemalloc? It's not
currently enabled on FreeBSD, but as far as I know, the only issue keeping it from
being useful is the absence of a Linux-compatible /proc/<pid>/maps (and the
gperftools folks may already have a solution for that; I haven't looked). I think it
makes more sense to get that sorted out than to develop a separate trace-based leak
checker. The problem with tracing is that it doesn't scale beyond some relatively
small number of allocator events.
Ok, we are in agreement on this all.
Paul Saab recommended profiling to me, but yes, the problem is that none
of this stuff works on FreeBSD out of the box due to missing bits here
or there. Augmenting the existing utrace stuff to get what I needed
seemed much simpler than figuring out how to get dtrace, pidmaps and
whatnot into the system. It's a matter of the requirements to
accomplish these higher order things requires
X=(skill+time+ability_to_socialize_these_changes) where X >
alfred->skill_and_time_and_socialize(). :)
To be honest, if dtrace just worked, then I could get the same
information I'm getting from utrace2(2) from dtrace with no problem.
(at least I think so).
As far as scaling it, I agree it does not work for long running
programs, however there are a few instances of programs leaking large
memory in a short while that I can track down by temporarily ktracing
for short while.
Is it time to start installing with some form of debug symbols? This would help
us also with dtrace.
Re: debug symbols, frame pointers, etc. necessary to make userland dtrace work
by default, IMO we should strongly prefer such defaults. It's more reasonable
to expect people who need every last bit of performance to remove functionality
than to expect people who want to figure out what the system is doing to figure
out what functionality to turn on.
Yes!!! :)
Is there an easy way to go about this?
Rui says it's really a matter of just turning off stripping of shlibs
and adding -fno-omit-frame-pointer and WITH_CTF.
I'm going to give this a shot, if it works, can you help me refine this?
I'll post diffs later today if I don't get completely stuck somehow.
-Alfred
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
