> From: [EMAIL PROTECTED]
> [mailto:[EMAIL PROTECTED]]On Behalf Of Jeff Trawick
> Subject: Re: cvs commit: httpd-2.0 STATUS
> 
> "Sander Striker" <[EMAIL PROTECTED]> writes:
> 
>>> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]
>>> Subject: cvs commit: httpd-2.0 STATUS
>>> 
>>> trawick     02/01/13 07:56:29
>>> 
>>>   Modified:    .        STATUS
>>>   Log:
>>>   Brian's fix looks good!
>>>   
>>>   with Brian's fixed applied and running the same tests:
>>>   
>>>     I don't hit any segfaults with yesterday's HEAD and APR_POOL_DEBUG defined
>>>   
>>>     I don't hit any segfaults with current HEAD and APR_POOL_DEBUG *not defined*
>>>   
>>>     I do hit segfaults down in apr_pool debug code with current HEAD and
>>>     APR_POOL_DEBUG *defined*
>> 
>> Do you have a backtrace?  Are you using electric fence or not?
> 
> as mentioned previously, no electric fence...

Ok, just checking.
 
> I hit it pretty soon with one worker server process and lots of
> threads serving connections.  The only patch applied was uncommenting
> APR_POOL_DEBUG in apr_pools.h.  I'm running HEAD from approx. half an
> hour ago.
> 
> Here is one backtrace (I wasn't seeing this assertion failure until I
> updated about half an hour ago and picked up the latest apr_pools.c):
> 
> #0  0xff10c2f0 in _cleanup () from /usr/lib/libc.so.1
> (gdb) where
> #0  0xff10c2f0 in _cleanup () from /usr/lib/libc.so.1
> #1  0xff0b5110 in abort () from /usr/lib/libc.so.1
> #2  0xff34c82c in check_integrity (pool=0x172f68) at apr_pools.c:926
> #3  0xff34c84c in apr_palloc (pool=0x172f68, size=12) at
> apr_pools.c:939
> #4  0xff3862ac in apr_brigade_create (p=0x172f68) at apr_brigade.c:104
> #5  0xcdd10 in core_output_filter (f=0x18f9d0, b=0x22c898) at
> core.c:3408
> #6  0xbe5f0 in ap_pass_brigade (next=0x18f9d0, bb=0x1eae78) at
> util_filter.c:388
> #7  0x4ebe8 in ap_http_header_filter (f=0x1a9950, b=0x1eae78) at
> http_protocol.c:1303
> #8  0xbe5f0 in ap_pass_brigade (next=0x1a9950, bb=0x1eae78) at
> util_filter.c:388
> #9  0xc31cc in ap_content_length_filter (f=0x1a29e0, b=0x1eae78) at
> protocol.c:1010
> #10 0xbe5f0 in ap_pass_brigade (next=0x1a29e0, bb=0x1eae78) at
> util_filter.c:388
> #11 0x51dc4 in ap_byterange_filter (f=0x2abd70, bb=0x1eae78) at
> http_protocol.c:2460
> #12 0xbe5f0 in ap_pass_brigade (next=0x2abd70, bb=0x1eae78) at
> util_filter.c:388
> #13 0xcc388 in default_handler (r=0x2663b0) at core.c:2973
> #14 0xa6250 in ap_run_handler (r=0x2663b0) at config.c:185
> #15 0xa6ec8 in ap_invoke_handler (r=0x2663b0) at config.c:359
> #16 0x53b24 in ap_internal_redirect (new_uri=0x2ef4f8
> "/index.html.en", r=0x190ed0) at http_request.c:454
> #17 0x8bfa0 in handle_map_file (r=0x190ed0) at mod_negotiation.c:2838
> #18 0xa6250 in ap_run_handler (r=0x190ed0) at config.c:185
> #19 0xa6ec8 in ap_invoke_handler (r=0x190ed0) at config.c:359
> #20 0x533c0 in ap_process_request (r=0x190ed0) at http_request.c:292
> #21 0x4b1c0 in ap_process_http_connection (c=0x1909d8) at
> http_core.c:280
> #22 0xbab14 in ap_run_process_connection (c=0x1909d8) at
> connection.c:84
> #23 0xbb0cc in ap_process_connection (c=0x1909d8) at connection.c:230
> #24 0xa0b70 in process_socket (p=0x172f68, sock=0x172fb0,
> my_child_num=0, my_thread_num=20) at worker.c:562
> #25 0xa14b8 in worker_thread (thd=0x15a9f0, dummy=0x15ad78) at
> worker.c:777
> #26 0xff340c48 in dummy_worker (opaque=0x15a9f0) at thread.c:122

This backtrace shows that check_integrity is aborting on us.
That could mean that the pool passed to apr_palloc was destroyed
previously.

/me slaps himself...
Or it could mean that I was to stupid to lock the child list
before traversing it.  I'll patch that up.  Could you try again
after that?

> Here is another flavor:
> 
> Cannot access memory at address 0xff3e08f4
> #0  0xff34c708 in ?? ()
> (gdb) where
> #0  0xff34c708 in ?? ()
> #1  0xff34c74c in ?? ()
> #2  0xff34c74c in ?? ()
> #3  0xff34c818 in ?? ()
> #4  0xff34c84c in ?? ()
> #5  0xff34d9d0 in ?? ()
> #6  0xff336844 in ?? ()
> #7  0xcc0f4 in default_handler (r=0x147138) at core.c:2928
> #8  0xa6250 in ap_run_handler (r=0x147138) at config.c:185
> #9  0xa6ec8 in ap_invoke_handler (r=0x147138) at config.c:359
> #10 0xd3dd0 in ap_run_sub_req (r=0x147138) at request.c:1746
> #11 0x37208 in handle_include (ctx=0x1ab500, bb=0xf35016d8,
> r=0x2f9ce8, f=0x2c45e0, head_ptr=0x13c180,
>     inserted_head=0xf35015d4) at mod_include.c:1237
> #12 0x3cc00 in send_parsed_content (bb=0xf35016d8, r=0x2f9ce8,
> f=0x2c45e0) at mod_include.c:2996
> #13 0x3db98 in includes_filter (f=0x2c45e0, b=0x161b20) at
> mod_include.c:3270
> #14 0xbe5f0 in ap_pass_brigade (next=0x2c45e0, bb=0x15b590) at
> util_filter.c:388
> #15 0x8bec4 in handle_map_file (r=0x2f9ce8) at mod_negotiation.c:2824
> #16 0xa6250 in ap_run_handler (r=0x2f9ce8) at config.c:185
> #17 0xa6ec8 in ap_invoke_handler (r=0x2f9ce8) at config.c:359
> #18 0x53b24 in ap_internal_redirect (new_uri=0x1386c0
> "/error/HTTP_FORBIDDEN.html.var", r=0x1fb480)
>     at http_request.c:454
> #19 0x531e4 in ap_die (type=403, r=0x1fb480) at http_request.c:218
> #20 0x53414 in ap_process_request (r=0x1fb480) at http_request.c:304
> #21 0x4b1c0 in ap_process_http_connection (c=0x1fa780) at
> http_core.c:280
> #22 0xbab14 in ap_run_process_connection (c=0x1fa780) at
> connection.c:84
> #23 0xbb0cc in ap_process_connection (c=0x1fa780) at connection.c:230
> #24 0xa0b70 in process_socket (p=0x1a5d68, sock=0x1a5db0,
> my_child_num=0, my_thread_num=150) at worker.c:562
> #25 0xa14b8 in worker_thread (thd=0x155bd0, dummy=0x156748) at
> worker.c:777
> #26 0xff340c48 in ?? ()

This is badness.  Where did that memory address come from?

> Here is another flavor:
> 
> #0  0xff117acc in _poll () from /usr/lib/libc.so.1
> (gdb) where
> #0  0xff117acc in _poll () from /usr/lib/libc.so.1
> #1  0xff0cb2e8 in select () from /usr/lib/libc.so.1
> #2  0xff05b5b0 in select () from /usr/lib/libthread.so.1
> #3  0xff33aab0 in apr_recv (sock=0x1edf60, buf=0x1d0630 "",
> len=0xfb30d6e8) at sendrecv.c:142
> #4  0xff385814 in socket_read (a=0x1c9148, str=0xfb30d6ec,
> len=0xfb30d6e8, block=APR_BLOCK_READ)
>     at apr_buckets_socket.c:75
> #5  0xcd10c in core_input_filter (f=0x1c8088, b=0x189688,
> mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) at core.c:3156
> #6  0xbe520 in ap_get_brigade (next=0x1c8088, bb=0x189688,
> mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4)
>     at util_filter.c:362
> #7  0xcc690 in net_time_filter (f=0x1cc128, b=0x189688,
> mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4) at core.c:3002
> #8  0xbe520 in ap_get_brigade (next=0x1cc128, bb=0x189688,
> mode=AP_MODE_BLOCKING, readbytes=0xfb30d8d4)
>     at util_filter.c:362
> #9  0xc0bf8 in ap_rgetline (s=0x193618, n=8192, r=0x193600, fold=0) at
> protocol.c:225
> #10 0xc14e0 in read_request_line (r=0x193600) at protocol.c:429
> #11 0xc1dc4 in ap_read_request (conn=0x18f230) at protocol.c:608
> #12 0x4b154 in ap_process_http_connection (c=0x18f230) at
> http_core.c:273
> #13 0xbab14 in ap_run_process_connection (c=0x18f230) at
> connection.c:84
> #14 0xbb0cc in ap_process_connection (c=0x18f230) at connection.c:230
> #15 0xa0b70 in process_socket (p=0x1edf18, sock=0x1edf60,
> my_child_num=0, my_thread_num=309) at worker.c:562
> #16 0xa14b8 in worker_thread (thd=0x162600, dummy=0x1573b0) at
> worker.c:777
> #17 0xff340c48 in dummy_worker (opaque=0x162600) at thread.c:122

Haven't got a clue about this one.

> Here finally is the one I mentioned previously:
> 
> Loaded symbols for /usr/lib/libthread.so.1
> #0  0xff34c708 in pool_is_child_of (pool=0x168f30, parent=0x2537b0) at
> apr_pools.c:898
> 898         child = parent->child;
> (gdb) where
> #0  0xff34c708 in pool_is_child_of (pool=0x168f30, parent=0x2537b0) at
> apr_pools.c:898
> #1  0xff34c74c in pool_is_child_of (pool=0x168f30, parent=0x235470) at
> apr_pools.c:901
> #2  0xff34c74c in pool_is_child_of (pool=0x168f30, parent=0x117a60) at
> apr_pools.c:901
> #3  0xff34c818 in check_integrity (pool=0x168f30) at apr_pools.c:915
> #4  0xff34d9b0 in apr_pool_cleanup_register (p=0x168f30,
> data=0x171a38,
>     plain_cleanup_fn=0xff336358 <apr_unix_file_cleanup>,
> child_cleanup_fn=0xff336358 <apr_unix_file_cleanup>)
>     at apr_pools.c:1524
> #5  0xff336844 in apr_file_open (new=0xf7508cf4,
>     fname=0x2cb390
> "/export/home/trawick/apacheinst/error/include/bottom.html", flag=33,
> perm=0, cont=0x168f30)
>     at open.c:175
> #6  0xcc0f4 in default_handler (r=0x156288) at core.c:2928
> #7  0xa6250 in ap_run_handler (r=0x156288) at config.c:185
> #8  0xa6ec8 in ap_invoke_handler (r=0x156288) at config.c:359
> #9  0xd3dd0 in ap_run_sub_req (r=0x156288) at request.c:1746
> #10 0x37208 in handle_include (ctx=0x19e4d0, bb=0xf75094c0,
> r=0x19f708, f=0x19dc90, head_ptr=0x176700,
>     inserted_head=0xf75093bc) at mod_include.c:1237
> #11 0x3cc00 in send_parsed_content (bb=0xf75094c0, r=0x19f708,
> f=0x19dc90) at mod_include.c:2996
> #12 0x3db98 in includes_filter (f=0x19dc90, b=0x1b1d18) at
> mod_include.c:3270
> #13 0xbe5f0 in ap_pass_brigade (next=0x19dc90, bb=0x2cb2b8) at
> util_filter.c:388
> #14 0x8bec4 in handle_map_file (r=0x19f708) at mod_negotiation.c:2824
> #15 0xa6250 in ap_run_handler (r=0x19f708) at config.c:185
> #16 0xa6ec8 in ap_invoke_handler (r=0x19f708) at config.c:359
> #17 0x53b24 in ap_internal_redirect (new_uri=0x1386c0
> "/error/HTTP_FORBIDDEN.html.var", r=0x19e340)
>     at http_request.c:454
> #18 0x531e4 in ap_die (type=403, r=0x19e340) at http_request.c:218
> #19 0x53b48 in ap_internal_redirect (new_uri=0x2abdb8
> "/index.html.en", r=0x1a7610) at http_request.c:455
> #20 0x8bfa0 in handle_map_file (r=0x1a7610) at mod_negotiation.c:2838
> #21 0xa6250 in ap_run_handler (r=0x1a7610) at config.c:185
> #22 0xa6ec8 in ap_invoke_handler (r=0x1a7610) at config.c:359
> #23 0x533c0 in ap_process_request (r=0x1a7610) at http_request.c:292
> #24 0x4b1c0 in ap_process_http_connection (c=0x1a7050) at
> http_core.c:280
> #25 0xbab14 in ap_run_process_connection (c=0x1a7050) at
> connection.c:84
> #26 0xbb0cc in ap_process_connection (c=0x1a7050) at connection.c:230
> #27 0xa0b70 in process_socket (p=0x192c60, sock=0x192ca8,
> my_child_num=0, my_thread_num=360) at worker.c:562
> #28 0xa14b8 in worker_thread (thd=0x1607c8, dummy=0x151630) at
> worker.c:777
> #29 0xff340c48 in dummy_worker (opaque=0x1607c8) at thread.c:122

This sounds like the same lack of locking as the first bt.

> I get the feeling that I can create as many different backtraces as I
> feel like cutting and pasting.  Generally, about a second after I
> start pounding on the server it dumps core.

Well, thanks for supplying them. :)
I'll go and fix one of the problems.  Could you try again later?

> Jeff Trawick | [EMAIL PROTECTED] | PGP public key at web site:

Sander

Reply via email to