Crash inside worker mpm's ap_mpm_pod_check()

2010-12-13 Thread Graham Leggett

Hi all,

I am currently trying to get to the bottom of a crash that is  
occurring under load (during an Avalanche load test, most  
specifically) inside the worker mpm, with the stacktrace as below.


Most specifically, it segfaults inside the read() below:

AP_DECLARE(int) ap_mpm_pod_check(ap_pod_t *pod)
{
char c;
apr_os_file_t fd;
int rc;

/* we need to surface EINTR so we'll have to grab the
 * native file descriptor and do the OS read() ourselves
 */
apr_os_file_get(fd, pod-pod_in);
rc = read(fd, c, 1);

In theory, looking at the parameters of read(), c and 1 are  
calculated by the compiler, which leaves the possibility that the fd  
is invalid, although fd does seem to have a sensible value (4).


What does seem strange is this line in the backtrace:

#1  0x2aca224a5e77 in read (pod=value optimized out)

The parameters of read() are (int fd, void *buf, size_t nbytes), not  
(ap_pod_t *pod), this looks a bit odd.


The next step for me is to attempt to deploy a non optimised httpd to  
see if I can reproduce it, in the mean time, does this ring any bells  
with anybody?


(gdb) bt full
#0  0x2aca23d3491b in read () from /lib64/libpthread.so.0
No symbol table info available.
#1  0x2aca224a5e77 in read (pod=value optimized out)
at /usr/include/bits/unistd.h:35
No locals.
#2  ap_mpm_pod_check (pod=value optimized out)
at /usr/src/debug/httpd-2.2.17/server/mpm/worker/pod.c:54
c = value optimized out
fd = 4
#3  0x2aca224a3e42 in child_main (child_num_arg=value optimized  
out)

at /usr/src/debug/httpd-2.2.17/server/mpm/worker/worker.c:1258
threads = value optimized out
rv = 577498496
ts = 0x2aca2c4883f8
thread_attr = 0x2aca2c488418
start_thread_id = value optimized out
#4  0x2aca224a3ff5 in make_child (s=0x2aca287d5848, slot=2)
at /usr/src/debug/httpd-2.2.17/server/mpm/worker/worker.c:1341
pid = 0
#5  0x2aca224a40af in startup_children (number_to_start=8)
at /usr/src/debug/httpd-2.2.17/server/mpm/worker/worker.c:1375
i = 2
#6  0x2aca224a4b7f in ap_mpm_run (_pconf=0x2aca287ce138,
---Type return to continue, or q return to quit---
plog=value optimized out, s=0x2aca287d5848)
at /usr/src/debug/httpd-2.2.17/server/mpm/worker/worker.c:1742
remaining_children_to_start = 10
rv = value optimized out
#7  0x2aca2247dc73 in main (argc=4, argv=0x7fff25e2ca78)
at /usr/src/debug/httpd-2.2.17/server/main.c:739
c = 68 'D'
configtestonly = 0
confname = 0x2aca224a7478 conf/httpd.conf
def_server_root = 0x2aca224a7488 /etc/httpd
temp_error_log = 0x0
error = value optimized out
process = 0x2aca287cc218
server_conf = value optimized out
pglobal = 0x2aca287cc128
pconf = 0x2aca287ce138
plog = 0x2aca288002c8
ptemp = 0x2aca287d2158
pcommands = 0x2aca287d0148
opt = 0x2aca287d0238
rv = value optimized out
optarg = 0x7fff25e2cf32 SERVER_CLASS_STATICP
(gdb)

Regards,
Graham
--



Re: Crash inside worker mpm's ap_mpm_pod_check()

2010-12-13 Thread Jeff Trawick
On Mon, Dec 13, 2010 at 8:06 PM, Graham Leggett minf...@sharp.fm wrote:
 Hi all,

 I am currently trying to get to the bottom of a crash that is occurring
 under load (during an Avalanche load test, most specifically) inside the
 worker mpm, with the stacktrace as below.

 Most specifically, it segfaults inside the read() below:

Check the other threads.  It didn't segfault on this one.

If gdb doesn't properly identify the crasher, find the threads that
that aren't in a syscall; one of those is the culprit.