...looks like a problem with cleaning up an mmap bucket. This is from
/usr/local/apache2.0.35/corefiles/httpd.core.3 ; .4 and .5 are the same problem.
#0 apr_pool_cleanup_kill (p=0x8152f08, data=0x8152eb8,
cleanup_fn=0x280cc700 <mmap_cleanup>) at apr_pools.c:1669
#1 0x280cc90a in apr_mmap_delete (mm=0x8152eb8) at mmap.c:210
#2 0x280ad926 in mmap_destroy (data=0x8131298) at apr_buckets_mmap.c:82
#3 0x280adf08 in apr_brigade_cleanup (data=0x8134ca8) at apr_brigade.c:86
#4 0x280adebe in brigade_cleanup (data=0x8134ca8) at apr_brigade.c:72
#5 0x280cdd3b in run_cleanups (c=0x813cb98) at apr_pools.c:1713
#6 0x280cd51c in apr_pool_destroy (pool=0x814b010) at apr_pools.c:638
#7 0x280cd417 in apr_pool_clear (pool=0x812c010) at apr_pools.c:600
#8 0x8064752 in child_main (child_num_arg=291) at prefork.c:586
the mm looks whacked/previously deleted:
(gdb) fr 1
#1 0x280cc90a in apr_mmap_delete (mm=0x8152eb8) at mmap.c:210
(gdb) p *mm
$1 = {cntxt = 0x8152f08, mm = 0x0, size = 0, is_owner = 0}
(gdb) p *mm.cntxt
$2 = {parent = 0x8151010, child = 0x10, sibling = 0x2, ref = 0x28,
cleanups = 0x8147010, allocator = 0x0, subprocesses = 0x0, abort_fn = 0,
user_data = 0x0, tag = 0x0, active = 0x0, self = 0x0, self_first_avail = 0x0}
...but the bucket structures looks fine:
(gdb) fr 3
#3 0x280adf08 in apr_brigade_cleanup (data=0x8134ca8) at apr_brigade.c:86
86 apr_bucket_delete(e);
(gdb) dump_bucket e
bucket=MMAP(0x8131190), length=475, data=0x8131298
(gdb) p *e
$3 = {link = {next = 0x8134cac, prev = 0x8134cac}, type = 0x280b79f0,
length = 475, start = 0, data = 0x8131298,
free = 0x805cf18 <apr_bucket_free>, list = 0x8131010}
(gdb) p *(apr_bucket_mmap *)e.data
$4 = {refcount = {refcount = 0}, mmap = 0x8152eb8}
If you have an account on daedalus and you want to take a look,
cd /usr/local/apache2.0.35
gdb bin/httpd corefiles/httpd.core.[2..5]
Dump 2 is a different. It shows up in sendfile intermittently. We hit
something similar about a month ago. This one needs a trap to prove whether or
not we're passing Good Stuff to the FreeBSD kernel.
Greg