One more backtrace when the breakpoints were set to both ENGINE_load_builtin_engines and ENGINE_cleanup
The apache mod-ssl tries to ENGINE_cleanup after the first ENGINE_load_builtin_engines call. So it looks as if the cleanup wasn't successful.
GNU gdb (GDB) 7.1-ubuntu Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/sbin/apache2...done. (gdb) handle SIG33 pass nostop noprint Signal Stop Print Pass to program Description SIG33 No No Yes Real-time event 33 (gdb) set pagination 0 (gdb) break ENGINE_load_builtin_engines Function "ENGINE_load_builtin_engines" not defined. Make breakpoint pending on future shared library load? (y or [n]) Breakpoint 1 (ENGINE_load_builtin_engines) pending. (gdb) break ENGINE_cleanup Function "ENGINE_cleanup" not defined. Make breakpoint pending on future shared library load? (y or [n]) Breakpoint 2 (ENGINE_cleanup) pending. (gdb) run Starting program: /usr/sbin/apache2 -k start -X [Thread debugging using libthread_db enabled] Breakpoint 1, ENGINE_load_builtin_engines () at eng_all.c:72 72 ENGINE_load_padlock(); (gdb) backtrace #0 ENGINE_load_builtin_engines () at eng_all.c:72 #1 0x00007ffff115af65 in ssl_hook_pre_config (pconf=0x7ffff820e138, plog=<value optimized out>, ptemp=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/ssl/mod_ssl.c:274 #2 0x00007ffff7fd5e7e in ap_run_pre_config (pconf=0x7ffff820e138, plog=0x7ffff82422d8, ptemp=0x7ffff82442e8) at /build/buildd/apache2-2.2.14/server/config.c:86 #3 0x00007ffff7fc2120 in main (argc=4, argv=0x7fffffffe728) at /build/buildd/apache2-2.2.14/server/main.c:640 (gdb) c Continuing. Breakpoint 2, ENGINE_cleanup () at eng_lib.c:188 188 if(int_cleanup_check(0)) (gdb) backtrace #0 ENGINE_cleanup () at eng_lib.c:188 #1 0x00007ffff115afb8 in ssl_cleanup_pre_config (data=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/ssl/mod_ssl.c:238 #2 0x00007ffff7706c23 in ?? () from /usr/lib/libapr-1.so.0 #3 0x00007ffff77059af in apr_pool_clear () from /usr/lib/libapr-1.so.0 #4 0x00007ffff7fc21fd in main (argc=4, argv=0x7fffffffe728) at /build/buildd/apache2-2.2.14/server/main.c:692 (gdb) c Continuing. warning: Temporarily disabling breakpoints for unloaded shared library "/lib/libcrypto.so.0.9.8" Breakpoint 1, ENGINE_load_builtin_engines () at eng_all.c:72 72 ENGINE_load_padlock(); (gdb) backtrace #0 ENGINE_load_builtin_engines () at eng_all.c:72 #1 0x00007ffff115af65 in ssl_hook_pre_config (pconf=0x7ffff820e138, plog=<value optimized out>, ptemp=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/ssl/mod_ssl.c:274 #2 0x00007ffff7fd5e7e in ap_run_pre_config (pconf=0x7ffff820e138, plog=0x7ffff82422d8, ptemp=0x7ffff8216178) at /build/buildd/apache2-2.2.14/server/config.c:86 #3 0x00007ffff7fc2298 in main (argc=4, argv=0x7fffffffe728) at /build/buildd/apache2-2.2.14/server/main.c:711 (gdb) c Continuing. Breakpoint 1, ENGINE_load_builtin_engines () at eng_all.c:72 72 ENGINE_load_padlock(); (gdb) backtrace #0 ENGINE_load_builtin_engines () at eng_all.c:72 #1 0x00007ffff0088c09 in ?? () from /usr/lib/libcurl.so.4 #2 0x00007ffff0091b59 in curl_global_init () from /usr/lib/libcurl.so.4 #3 0x00007ffff02b66f0 in zm_startup_curl (type=1, module_number=43) at /build/buildd/php5-5.3.2/ext/curl/interface.c:819 #4 0x00007ffff3d7bbcb in zend_startup_module_ex (module=0x7ffff84ce8a0) at /build/buildd/php5-5.3.2/Zend/zend_API.c:1613 #5 0x00007ffff3d84865 in zend_hash_apply (ht=0x7ffff44701c0, apply_func=0x7ffff3d7bad0 <zend_startup_module_ex>) at /build/buildd/php5-5.3.2/Zend/zend_hash.c:875 #6 0x00007ffff3d7ef4a in zend_startup_modules () at /build/buildd/php5-5.3.2/Zend/zend_API.c:1662 #7 0x00007ffff3d254b5 in php_module_startup (sf=0x7ffff446f120, additional_modules=0x7ffff74c7e40, num_additional_modules=1) at /build/buildd/php5-5.3.2/main/main.c:2050 #8 0x00007ffff3e043b5 in php_apache2_startup (sapi_module=0x1) at /build/buildd/php5-5.3.2/sapi/apache2handler/sapi_apache2.c:335 #9 0x00007ffff3e04fa0 in php_apache_server_startup (pconf=0x7ffff820e138, plog=0x7fffffffe340, ptemp=0x7ffff71c7360, s=0x7ffff8214938) at /build/buildd/php5-5.3.2/sapi/apache2handler/sapi_apache2.c:448 #10 0x00007ffff7fd5f79 in ap_run_post_config (pconf=0x7ffff820e138, plog=0x7ffff82422d8, ptemp=0x7ffff8216178, s=0x7ffff8214938) at /build/buildd/apache2-2.2.14/server/config.c:95 #11 0x00007ffff7fc2327 in main (argc=4, argv=0x7fffffffe728) at /build/buildd/apache2-2.2.14/server/main.c:731 (gdb) c Continuing. [New Thread 0x7fffe8d5d710 (LWP 3488)] [Thread 0x7fffe8d5d710 (LWP 3488) exited] Program received signal SIGSEGV, Segmentation fault. 0x00007ffff214ea60 in sha1_md () from /lib/libcrypto.so.0.9.8 (gdb) backtrace #0 0x00007ffff214ea60 in sha1_md () from /lib/libcrypto.so.0.9.8 #1 0x00007ffff1e7c7cf in engine_unlocked_init (e=0x7ffff86411b0) at eng_init.c:67 #2 0x00007ffff1e7dc34 in engine_table_select (table=0x7ffff216c860, nid=427) at eng_table.c:274 #3 0x00007ffff1e7f095 in ENGINE_get_cipher_engine (nid=427) at tb_cipher.c:115 #4 0x00007ffff1e96ad6 in do_evp_enc_engine (ctx=0x7ffff86f6f10, pcipher=0x7fffffffda30, impl=0x0) at enc_min.c:161 #5 0x00007ffff1e96c24 in EVP_CipherInit_ex (ctx=0x7ffff86f6f10, cipher=0x7ffff214a0e0, impl=0x0, key=0x7ffff86f6ea8 "\035c|\312!\235S\271\264GG\314?ڥ.jp\240JՈ\221\063E1H\354 z\036\027\223\310\b\213\350\226Υ\261bF\255\353\\\365\353}\023", iv=0x7ffff86f6ee8 "=\304\\M9!\032\a\263\313\002\230\230\020\263\023C\273\204\233\360\352\312\f[\267\021\tU\231\070", <incomplete sequence \347\261>, enc=0) at enc_min.c:229 #6 0x00007ffff382f686 in tls1_change_cipher_state (s=0x7ffff86d6c10, which=33) at t1_enc.c:434 #7 0x00007ffff3828cbe in ssl3_do_change_cipher_spec (s=0x7ffff86d6c10) at s3_pkt.c:1246 #8 0x00007ffff382889b in ssl3_read_bytes (s=0x7ffff86d6c10, type=22, buf=0x7ffff86df430 "\020", len=4, peek=0) at s3_pkt.c:1110 #9 0x00007ffff3829ad9 in ssl3_get_message (s=0x7ffff86d6c10, st1=8608, stn=8609, mt=-1, max=514, ok=0x7fffffffde28) at s3_both.c:394 #10 0x00007ffff381c12b in ssl3_get_cert_verify (s=0x7ffff86d6c10) at s3_srvr.c:2311 #11 0x00007ffff381895b in ssl3_accept (s=0x7ffff86d6c10) at s3_srvr.c:506 #12 0x00007ffff383bf64 in SSL_accept (s=0x7ffff86d6c10) at ssl_lib.c:869 #13 0x00007ffff382b199 in ssl23_get_client_hello (s=0x7ffff86d6c10) at s23_srvr.c:577 #14 0x00007ffff382a479 in ssl23_accept (s=0x7ffff86d6c10) at s23_srvr.c:203 #15 0x00007ffff383bf64 in SSL_accept (s=0x7ffff86d6c10) at ssl_lib.c:869 #16 0x00007ffff1160ab8 in ssl_io_filter_connect (filter_ctx=0x7ffff86d33b8) at /build/buildd/apache2-2.2.14/modules/ssl/ssl_engine_io.c:1103 #17 0x00007ffff1161a38 in ssl_io_filter_input (f=0x7ffff86dc408, bb=0x7ffff86dea28, mode=<value optimized out>, block=APR_BLOCK_READ, readbytes=<value optimized out>) at /build/buildd/apache2-2.2.14/modules/ssl/ssl_engine_io.c:1349 #18 0x00007ffff7fc9626 in ap_rgetline_core (s=0x7ffff86dd4f8, n=<value optimized out>, read=<value optimized out>, r=<value optimized out>, fold=<value optimized out>, bb=<value optimized out>) at /build/buildd/apache2-2.2.14/server/protocol.c:231 #19 0x00007ffff7fc9fa6 in read_request_line (conn=0x7ffff86d2af8) at /build/buildd/apache2-2.2.14/server/protocol.c:596 #20 ap_read_request (conn=0x7ffff86d2af8) at /build/buildd/apache2-2.2.14/server/protocol.c:891 #21 0x00007ffff7fe4490 in ap_process_http_connection (c=0x7ffff86d2af8) at /build/buildd/apache2-2.2.14/modules/http/http_core.c:183 #22 0x00007ffff7fddcf8 in ap_run_process_connection (c=0x7ffff86d2af8) at /build/buildd/apache2-2.2.14/server/connection.c:43 #23 0x00007ffff7fec037 in child_main (child_num_arg=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:662 #24 0x00007ffff7fec306 in make_child (s=0x7ffff8214938, slot=0) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:702 #25 0x00007ffff7fec953 in ap_mpm_run (_pconf=<value optimized out>, plog=<value optimized out>, s=<value optimized out>) at /build/buildd/apache2-2.2.14/server/mpm/prefork/prefork.c:978 #26 0x00007ffff7fc2350 in main (argc=4, argv=0x7fffffffe728) at /build/buildd/apache2-2.2.14/server/main.c:742 (gdb) q A debugging session is active. Inferior 1 [process 3401] will be killed. Quit anyway? (y or n)
