> From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED]]On Behalf Of Jeff Trawick > Subject: Re: cvs commit: httpd-2.0 STATUS > > "Sander Striker" <[EMAIL PROTECTED]> writes: > >>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] >>> Subject: cvs commit: httpd-2.0 STATUS >>> >>> trawick 02/01/13 07:56:29 >>> >>> Modified: . STATUS >>> Log: >>> Brian's fix looks good! >>> >>> with Brian's fixed applied and running the same tests: >>> >>> I don't hit any segfaults with yesterday's HEAD and APR_POOL_DEBUG defined >>> >>> I don't hit any segfaults with current HEAD and APR_POOL_DEBUG *not defined* >>> >>> I do hit segfaults down in apr_pool debug code with current HEAD and >>> APR_POOL_DEBUG *defined* >> >> Do you have a backtrace? Are you using electric fence or not? > > as mentioned previously, no electric fence...
Ok, just checking. > I hit it pretty soon with one worker server process and lots of > threads serving connections. The only patch applied was uncommenting > APR_POOL_DEBUG in apr_pools.h. I'm running HEAD from approx. half an > hour ago. > > Here is one backtrace (I wasn't seeing this assertion failure until I > updated about half an hour ago and picked up the latest apr_pools.c): > > #0 0xff10c2f0 in _cleanup () from /usr/lib/libc.so.1 > (gdb) where > #0 0xff10c2f0 in _cleanup () from /usr/lib/libc.so.1 > #1 0xff0b5110 in abort () from /usr/lib/libc.so.1 > #2 0xff34c82c in check_integrity (pool=0x172f68) at apr_pools.c:926 > #3 0xff34c84c in apr_palloc (pool=0x172f68, size=12) at > apr_pools.c:939 > #4 0xff3862ac in apr_brigade_create (p=0x172f68) at apr_brigade.c:104 > #5 0xcdd10 in core_output_filter (f=0x18f9d0, b=0x22c898) at > core.c:3408 > #6 0xbe5f0 in ap_pass_brigade (next=0x18f9d0, bb=0x1eae78) at > util_filter.c:388 > #7 0x4ebe8 in ap_http_header_filter (f=0x1a9950, b=0x1eae78) at > http_protocol.c:1303 > #8 0xbe5f0 in ap_pass_brigade (next=0x1a9950, bb=0x1eae78) at > util_filter.c:388 > #9 0xc31cc in ap_content_length_filter (f=0x1a29e0, b=0x1eae78) at > protocol.c:1010 > #10 0xbe5f0 in ap_pass_brigade (next=0x1a29e0, bb=0x1eae78) at > util_filter.c:388 > #11 0x51dc4 in ap_byterange_filter (f=0x2abd70, bb=0x1eae78) at > http_protocol.c:2460 > #12 0xbe5f0 in ap_pass_brigade (next=0x2abd70, bb=0x1eae78) at > util_filter.c:388 > #13 0xcc388 in default_handler (r=0x2663b0) at core.c:2973 > #14 0xa6250 in ap_run_handler (r=0x2663b0) at config.c:185 > #15 0xa6ec8 in ap_invoke_handler (r=0x2663b0) at config.c:359 > #16 0x53b24 in ap_internal_redirect (new_uri=0x2ef4f8 > "/index.html.en", r=0x190ed0) at http_request.c:454 > #17 0x8bfa0 in handle_map_file (r=0x190ed0) at mod_negotiation.c:2838 > #18 0xa6250 in ap_run_handler (r=0x190ed0) at config.c:185 > #19 0xa6ec8 in ap_invoke_handler (r=0x190ed0) at config.c:359 > #20 0x533c0 in ap_process_request (r=0x190ed0) at http_request.c:292 > #21 0x4b1c0 in ap_process_http_connection (c=0x1909d8) at > http_core.c:280 > #22 0xbab14 in ap_run_process_connection (c=0x1909d8) at > connection.c:84 > #23 0xbb0cc in ap_process_connection (c=0x1909d8) at connection.c:230 > #24 0xa0b70 in process_socket (p=0x172f68, sock=0x172fb0, > my_child_num=0, my_thread_num=20) at worker.c:562 > #25 0xa14b8 in worker_thread (thd=0x15a9f0, dummy=0x15ad78) at > worker.c:777 > #26 0xff340c48 in dummy_worker (opaque=0x15a9f0) at thread.c:122 This backtrace shows that check_integrity is aborting on us. That could mean that the pool passed to apr_palloc was destroyed previously. /me slaps himself... Or it could mean that I was to stupid to lock the child list before traversing it. I'll patch that up. Could you try again after that? > Here is another flavor: > > Cannot access memory at address 0xff3e08f4 > #0 0xff34c708 in ?? () > (gdb) where > #0 0xff34c708 in ?? () > #1 0xff34c74c in ?? () > #2 0xff34c74c in ?? () > #3 0xff34c818 in ?? () > #4 0xff34c84c in ?? () > #5 0xff34d9d0 in ?? () > #6 0xff336844 in ?? () > #7 0xcc0f4 in default_handler (r=0x147138) at core.c:2928 > #8 0xa6250 in ap_run_handler (r=0x147138) at config.c:185 > #9 0xa6ec8 in ap_invoke_handler (r=0x147138) at config.c:359 > #10 0xd3dd0 in ap_run_sub_req (r=0x147138) at request.c:1746 > #11 0x37208 in handle_include (ctx=0x1ab500, bb=0xf35016d8, > r=0x2f9ce8, f=0x2c45e0, head_ptr=0x13c180, > inserted_head=0xf35015d4) at mod_include.c:1237 > #12 0x3cc00 in send_parsed_content (bb=0xf35016d8, r=0x2f9ce8, > f=0x2c45e0) at mod_include.c:2996 > #13 0x3db98 in includes_filter (f=0x2c45e0, b=0x161b20) at > mod_include.c:3270 > #14 0xbe5f0 in ap_pass_brigade (next=0x2c45e0, bb=0x15b590) at > util_filter.c:388 > #15 0x8bec4 in handle_map_file (r=0x2f9ce8) at mod_negotiation.c:2824 > #16 0xa6250 in ap_run_handler (r=0x2f9ce8) at config.c:185 > #17 0xa6ec8 in ap_invoke_handler (r=0x2f9ce8) at config.c:359 > #18 0x53b24 in ap_internal_redirect (new_uri=0x1386c0 > "/error/HTTP_FORBIDDEN.html.var", r=0x1fb480) > at http_request.c:454 > #19 0x531e4 in ap_die (type=403, r=0x1fb480) at http_request.c:218 > #20 0x53414 in ap_process_request (r=0x1fb480) at http_request.c:304 > #21 0x4b1c0 in ap_process_http_connection (c=0x1fa780) at > http_core.c:280 > #22 0xbab14 in ap_run_process_connection (c=0x1fa780) at > connection.c:84 > #23 0xbb0cc in ap_process_connection (c=0x1fa780) at connection.c:230 > #24 0xa0b70 in process_socket (p=0x1a5d68, sock=0x1a5db0, > my_child_num=0, my_thread_num=150) at worker.c:562 > #25 0xa14b8 in worker_thread (thd=0x155bd0, dummy=0x156748) at > worker.c:777 > #26 0xff340c48 in ?? () This is badness. Where did that memory address come from? > Here is another flavor: > > #0 0xff117acc in _poll () from /usr/lib/libc.so.1 > (gdb) where > #0 0xff117acc in _poll () from /usr/lib/libc.so.1 > #1 0xff0cb2e8 in select () from /usr/lib/libc.so.1 > #2 0xff05b5b0 in select () from /usr/lib/libthread.so.1 > #3 0xff33aab0 in apr_recv (sock=0x1edf60, buf=0x1d0630 "", > len=0xfb30d6e8) at sendrecv.c:142 > #4 0xff385814 in socket_read (a=0x1c9148, str=0xfb30d6ec, > len=0xfb30d6e8, block=APR_BLOCK_READ) > at apr_buckets_socket.c:75 > #5 0xcd10c in core_input_filter (f=0x1c8088, b=0x189688, > mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) at core.c:3156 > #6 0xbe520 in ap_get_brigade (next=0x1c8088, bb=0x189688, > mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) > at util_filter.c:362 > #7 0xcc690 in net_time_filter (f=0x1cc128, b=0x189688, > mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) at core.c:3002 > #8 0xbe520 in ap_get_brigade (next=0x1cc128, bb=0x189688, > mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) > at util_filter.c:362 > #9 0xc0bf8 in ap_rgetline (s=0x193618, n=8192, r=0x193600, fold=0) at > protocol.c:225 > #10 0xc14e0 in read_request_line (r=0x193600) at protocol.c:429 > #11 0xc1dc4 in ap_read_request (conn=0x18f230) at protocol.c:608 > #12 0x4b154 in ap_process_http_connection (c=0x18f230) at > http_core.c:273 > #13 0xbab14 in ap_run_process_connection (c=0x18f230) at > connection.c:84 > #14 0xbb0cc in ap_process_connection (c=0x18f230) at connection.c:230 > #15 0xa0b70 in process_socket (p=0x1edf18, sock=0x1edf60, > my_child_num=0, my_thread_num=309) at worker.c:562 > #16 0xa14b8 in worker_thread (thd=0x162600, dummy=0x1573b0) at > worker.c:777 > #17 0xff340c48 in dummy_worker (opaque=0x162600) at thread.c:122 Haven't got a clue about this one. > Here finally is the one I mentioned previously: > > Loaded symbols for /usr/lib/libthread.so.1 > #0 0xff34c708 in pool_is_child_of (pool=0x168f30, parent=0x2537b0) at > apr_pools.c:898 > 898 child = parent->child; > (gdb) where > #0 0xff34c708 in pool_is_child_of (pool=0x168f30, parent=0x2537b0) at > apr_pools.c:898 > #1 0xff34c74c in pool_is_child_of (pool=0x168f30, parent=0x235470) at > apr_pools.c:901 > #2 0xff34c74c in pool_is_child_of (pool=0x168f30, parent=0x117a60) at > apr_pools.c:901 > #3 0xff34c818 in check_integrity (pool=0x168f30) at apr_pools.c:915 > #4 0xff34d9b0 in apr_pool_cleanup_register (p=0x168f30, > data=0x171a38, > plain_cleanup_fn=0xff336358 <apr_unix_file_cleanup>, > child_cleanup_fn=0xff336358 <apr_unix_file_cleanup>) > at apr_pools.c:1524 > #5 0xff336844 in apr_file_open (new=0xf7508cf4, > fname=0x2cb390 > "/export/home/trawick/apacheinst/error/include/bottom.html", flag=33, > perm=0, cont=0x168f30) > at open.c:175 > #6 0xcc0f4 in default_handler (r=0x156288) at core.c:2928 > #7 0xa6250 in ap_run_handler (r=0x156288) at config.c:185 > #8 0xa6ec8 in ap_invoke_handler (r=0x156288) at config.c:359 > #9 0xd3dd0 in ap_run_sub_req (r=0x156288) at request.c:1746 > #10 0x37208 in handle_include (ctx=0x19e4d0, bb=0xf75094c0, > r=0x19f708, f=0x19dc90, head_ptr=0x176700, > inserted_head=0xf75093bc) at mod_include.c:1237 > #11 0x3cc00 in send_parsed_content (bb=0xf75094c0, r=0x19f708, > f=0x19dc90) at mod_include.c:2996 > #12 0x3db98 in includes_filter (f=0x19dc90, b=0x1b1d18) at > mod_include.c:3270 > #13 0xbe5f0 in ap_pass_brigade (next=0x19dc90, bb=0x2cb2b8) at > util_filter.c:388 > #14 0x8bec4 in handle_map_file (r=0x19f708) at mod_negotiation.c:2824 > #15 0xa6250 in ap_run_handler (r=0x19f708) at config.c:185 > #16 0xa6ec8 in ap_invoke_handler (r=0x19f708) at config.c:359 > #17 0x53b24 in ap_internal_redirect (new_uri=0x1386c0 > "/error/HTTP_FORBIDDEN.html.var", r=0x19e340) > at http_request.c:454 > #18 0x531e4 in ap_die (type=403, r=0x19e340) at http_request.c:218 > #19 0x53b48 in ap_internal_redirect (new_uri=0x2abdb8 > "/index.html.en", r=0x1a7610) at http_request.c:455 > #20 0x8bfa0 in handle_map_file (r=0x1a7610) at mod_negotiation.c:2838 > #21 0xa6250 in ap_run_handler (r=0x1a7610) at config.c:185 > #22 0xa6ec8 in ap_invoke_handler (r=0x1a7610) at config.c:359 > #23 0x533c0 in ap_process_request (r=0x1a7610) at http_request.c:292 > #24 0x4b1c0 in ap_process_http_connection (c=0x1a7050) at > http_core.c:280 > #25 0xbab14 in ap_run_process_connection (c=0x1a7050) at > connection.c:84 > #26 0xbb0cc in ap_process_connection (c=0x1a7050) at connection.c:230 > #27 0xa0b70 in process_socket (p=0x192c60, sock=0x192ca8, > my_child_num=0, my_thread_num=360) at worker.c:562 > #28 0xa14b8 in worker_thread (thd=0x1607c8, dummy=0x151630) at > worker.c:777 > #29 0xff340c48 in dummy_worker (opaque=0x1607c8) at thread.c:122 This sounds like the same lack of locking as the first bt. > I get the feeling that I can create as many different backtraces as I > feel like cutting and pasting. Generally, about a second after I > start pounding on the server it dumps core. Well, thanks for supplying them. :) I'll go and fix one of the problems. Could you try again later? > Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site: Sander