On Thu, Feb 26, 2009 at 09:07:00PM +0100, max at bruningsystems.com wrote: > Jonathan Adams wrote: > >On Thu, Feb 26, 2009 at 11:42:59AM -0800, Edward Pilatowicz wrote: > > > >>On Thu, Feb 26, 2009 at 01:42:24PM -0500, James Carlson wrote: > >> > >>>I just spent a little over a day debugging a stack overflow problem in > >>>mdb itself. It turned out to be a fairly simple problem -- I'd added > >>>a new dcmd, and one of the functions had a structure on the stack that > >>>turned out to be unexpectedly _huge_ (512K+) -- but the symptoms of > >>>the problem were fairly misleading and unexpected. I saw panics that > >>>looked like this: > >>> > >>>kmdb ABORT: "../common/umem.c", line 1264: assertion failed: > >>>sp->slab_cache == cp > >>>Debugger aborted > >>>Program terminated > >>>{2} ok boot > >>> > >>>It turns out that allocating big things on the stack inside mdb can be > >>>somewhat toxic. > >>> > >>>I fixed my problem by allocating the offending structure with > >>>mdb_alloc, but that begs a question: are there other instances of this > >>>problem hiding in here? Could this be near the root of weird problems > >>>like CR 6766866? > >>> > >>>It seems to me that the compiler must (obviously) know how much > >>>storage it's reserving for auto variables. Is there any way to find > >>>this out and enforce a limit? That wouldn't fix the problem of > >>>nesting too deeply (or just recursing), but it'd at least catch > >>>obvious blunders before they turn into lengthy trials. > >>> > >>> > >>iirc, at some point, someone had a tool which could look at a kernel > >>panic stack trace and tell you the stack usage of each frame. but that > >>is only for post mortem analysis. afaik we don't have any way of > >>getting this information from the compiler, and we don't have any tools > >>that can do assembly analysis to determine this information. > >> > > > >It's pretty easy to look for subtractions from %rsp in the code: > > > >dis /kernel/kmdb/amd64/genunix | grep 'subq.*rsp' | > > sed 's/\(.*\):.*subq.*\$\(0x[0-9a-f]*\),%rsp.*$/\1 \2' | > > while read func off; do printf "%5d %s\n" "$off" "$func"; > > done | sort -n | egrep -v ' walkers\+| dcmds+" > > > >gives: > > > >... > > 4392 threadlist+0x21 > > 6592 pfile_callback+0x1d > > 7336 vmem+0x1d > > 8504 calloutid+0x21 > > > >So calloutid is the largest stack user. Something similar will work > >for sparc. We could put something like this in the kmdb module build, > >and have a "max allowed" value (10k?) > > > >It wouldn't solve heavily recursive stuff, but maybe a guard page or two > >could > >protect against that. > > > There is a redzone page at the low end of all(?) kernel stacks. When it > is touched, > the system panics (better than overwriting other memory). > See the code for segkp_fault(0 in uts/common/vm/seg_kp.c.
But kmdb has it's own stacks. I don't know if they have redzone pages. Cheers, - jonathan