Hi Sanjeev,
I have attached a script that does some more processing on the dtrace
output to provide more information in the summary.
On Mon, Jun 30, 2008 at 7:04 PM, Sanjeev Bagewadi
<[EMAIL PROTECTED]> wrote:
>
> I have a simple script for the userland and the details are available on
> my blog :
> http://blogs.sun.com/sanjeevb/
>
> The script is fairly rudimentary and I have intentionally avoided any
> processing during collection.
> All the intelligence is in the postprocessing Perl script.
>
> Probably there is room for optimization but, need to explore them.
>
> Hope they are of use.
>
In the dtrace output using the dtrace script I see instances of malloc
for a pointer(at a location) happening more than once without a free.
See example below. What to make of this? How can malloc return a
pointer location that was already returned by an earlier call to
malloc?
(this is for an actual application, I donot have a sample program that
can reproduce this scenario)
malloc:return tid=1 ptr=<ptr-value> size=48
<ustack_1>
....few lines without free.....
malloc:return tid=1 ptr=<*same* ptr-value> size=86 <- same memory location
<different_ustack>
....few lines.....
free:entry ptr=<ptr-value>
I had used your dtrace script and had found it to be quite useful.
Since the logs that used to be collected were often huge, the level of
detail perl script output provided wasn't sufficient.
The attached script (python) does little more work.
Ofcourse some more optimizations and still better reporting is possible.
For ex: I see some stacks in our app leak sporadically under heavy
load due to call drops and alternate execution paths that get
triggered. Such stacks aren't *always leaky*. This information can be
captured in the output.
=======================================================
ERROR line: <lineno> : Freeing non-existant memory at <ptr-value>
LEAK <nnn> bytes leaked. Allocated at line <line1> ptr <ptr-value> is
re-alloced at line <line2>
......multiple lines as above.......
=======================================================
INFO Leaks along with stack information is as follows:
=======================================================
Stack position (line no.): <line1> <line2> <line3>...
Pointers: <ptr1> <ptr2> <ptr3> ...
Size leaked: <bytes1> <bytes2> <bytes3>
Total size leaked: sumof(<bytes1> <bytes2> <bytes3> ...)
N(stack executions): <no of times the leaky stack got executed>
STACK IS:
<actual-stack>
......multiple stack info as above....
In the malloc related query, I was referring to the output line "LEAK
<nnn> bytes leaked. Allocated at line <line1> ptr <ptr-value> is
re-alloced at line <line2>"
This output is when there are 2 mallocs at <line1> and <line2> in the
dtrace output without a intermediate free.
-Shiv
#=======================================================================
# Author : Shivakumar GN
# License: CDDL v1.0
#=======================================================================
import sys
import re
import time
start_time = time.strftime('%X %x %Z')
start_clock = time.clock()
if len(sys.argv)!=2:
print 'Incorrect usage. Usage: sys.argv[0] <logfilename>'
sys.exit(-1)
filename = sys.argv[1]
f=open(filename, 'r')
line_no=0
# 3 56696 malloc:return ptr=0xee195ff8 size=1
malloc_rexp = re.compile('(?P<cpuid>[0-9]*)[ \t]*(?P<probeid>[0-9]*)[
\t]*malloc:return ptr=(?P<ptr>[0-9a-z]*) size=(?P<size>[0-9a-z]*)')
free_rexp = re.compile('free:entry Ptr=(?P<ptr>[0-9a-z]*)')
ptr_linedict = {}
ptr_sizedict = {}
ptr_cpudict = {}
ptr_probedict = {}
ptr_stackdict = {}
ptr_value=''
nNonExistantFree=0
nNullFrees=0
nReallocLeaks=0
nNotFreeLeaks=0
nTotalAllocs=0
nTotalFrees=0
inMallocStack=0
stack=''
for line_txt in f:
line_no=line_no+1
#print '----->', line_no, line_txt,
match=malloc_rexp.search(line_txt)
if inMallocStack==1:
stack = stack+line_txt
if line_txt=='\n':
ptr_stackdict[ptr_value]=stack
inMallocStack=0
stack=''
if match:
#print line_no, 'malloc: ', match.groups()
nTotalAllocs=nTotalAllocs+1
ptr_value=match.group('ptr')
alloc_size=match.group('size')
inMallocStack=1
for ptr, line in ptr_linedict.iteritems():
if ptr==ptr_value:
print 'LEAK', ptr_sizedict[ptr_value], 'bytes
leaked. Allocated at line',line,'Ptr', ptr, 'is re-alloced at line', line_no
nReallocLeaks=nReallocLeaks+1
break
ptr_linedict[match.group('ptr')]=line_no
ptr_sizedict[match.group('ptr')]=int(match.group('size'))
ptr_cpudict[match.group('ptr')]=match.group('cpuid')
ptr_probedict[match.group('ptr')]=match.group('probeid')
continue
free_match=free_rexp.search(line_txt)
if free_match:
nTotalFrees=nTotalFrees+1
inMallocStack=0
ptr_value=free_match.group('ptr')
found=0
for ptr, line in ptr_linedict.iteritems():
if ptr==ptr_value:
found=1
break
if found==0:
if ptr_value=='0x0':
print "ERROR line:",line_no,': Freeing NULL
pointer', ptr_value
nNullFrees = nNullFrees+1
else:
print "ERROR line:",line_no,': Freeing
non-existant memory at', ptr_value
nNonExistantFree = nNonExistantFree+1
else:
del ptr_linedict[ptr_value]
del ptr_sizedict[ptr_value]
del ptr_stackdict[ptr_value]
if inMallocStack==1:
inMallocStack=0
ptr_stackdict[ptr_value]=stack
stack=''
stack_lines_dict = {}
stack_count_dict = {}
stack_sizes_dict = {}
stack_totalsize_dict = {}
stack_ptrs_dict = {}
for ptr, line in ptr_linedict.iteritems():
nNotFreeLeaks=nNotFreeLeaks+1
temp_stack=ptr_stackdict[ptr]
if stack_lines_dict.has_key(temp_stack):
stack_lines_dict[temp_stack].append(line)
stack_count_dict[temp_stack] = stack_count_dict[temp_stack] + 1
stack_sizes_dict[temp_stack].append(ptr_sizedict[ptr])
stack_totalsize_dict[temp_stack] =
stack_totalsize_dict[temp_stack] + ptr_sizedict[ptr]
stack_ptrs_dict[temp_stack].append(ptr)
else:
stack_lines_dict[temp_stack] = [line]
stack_count_dict[temp_stack] = 1
stack_sizes_dict[temp_stack] = [ptr_sizedict[ptr]]
stack_totalsize_dict[temp_stack] = ptr_sizedict[ptr]
stack_ptrs_dict[temp_stack]=[ptr]
print '======================================================='
print 'INFO Leaks along with stack information is as follows:'
print '======================================================='
for stack, lines in stack_lines_dict.iteritems():
print 'Stack position (line no.): ', lines
print 'Pointers: ', stack_ptrs_dict[stack]
print 'Size leaked: ', stack_sizes_dict[stack]
print 'Total size leaked: ', stack_totalsize_dict[stack]
print 'N(stack executions): ', stack_count_dict[stack]
print 'STACK IS: \n', stack
print 'nNonExistantFree =', nNonExistantFree, 'nNullFrees=', nNullFrees,
'nReallocLeaks=', nReallocLeaks, 'nNotFreeLeaks=', nNotFreeLeaks,
print 'nTotalAllocs=', nTotalAllocs, 'nTotalFrees=', nTotalFrees
end_time = time.strftime('%X %x %Z')
end_clock = time.clock()
print '#Start time:', start_time
print '#End time:', end_time
print '#Start clock:', start_clock
print '#End clock:', end_clock
_______________________________________________
dtrace-discuss mailing list
[email protected]
