Juergen Note that stack red-zones are typically already generated. So, you don't want to mess with that. The following plan will work in conjunction with the system stack red-zone.
What I would do is: At startup, call down, with a defined limit. Each call level should take the local storage that is expected during "normal" calls. If this fails, the original target could not be achieved anyway. The signal() handler can be introduced before this is done. If a trap occurs, less stack nesting can be tried. On the deepest nesting, return a pointer to a local stack variable. Of course, this pointer cannot be used after return, but we just want to know the *location* of that memory. Use mprotect() on that pointer to establish the red page. a SIGSEGV will then be generated if the stack extends to that point. signal() will capture the SIGSEGV. Use setjmp() and longjmp() to unwind the stack after the SIGSEGV. Now, you want to use sigaction() to supply a sigaction handler, which gets passed a ucontext_t, which allows access to the actual address that caused the fault, the registers, etc. Old POSIX definition had it defined such that the fault could be processed, and a return from the signal handler would restart the instruction. More recent POSIX simply marked this as "undefined behaviour". This will work with both grow-up and grow-down stacks, *and* on implementations with "software stack". Eg. should work on Z-series. However, it is *not* (and cannot be) POSIX compliant. POSIX only allows mprotect() on mmap() memory (which is not stack memory). Also, POSIX specifies that execution becomes undefined after SIGSEGV -- even if it is caught. I just ignore these issues... (old POSIX, and very helpful if writing debug routines, and binary translators etc.). FredW
