It's not a matter of compiling OpenSSL to be thread safe. Someone needs to update the TLS C code to call the right OpenSSL API functions on module initialization. In it's current state I don't see how the TLS module can safely call OpenSSL from a threaded context.
>From the Openssl docs: http://openssl.org/docs/crypto/threads.html#DESCRIPTION "OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, locking_function and threadid_func." The TLS C code doesn't setup either one of those callbacks, so that's a problem. I'm not sure if that is your problem specifically but it would be a good place to start. -Andrew On Fri, May 1, 2009 at 12:59 PM, Jade Rubick <[email protected]> wrote: > > Jade Rubick > Director of Development > TRUiST > 120 Wall Street, 4th Floor > New York, NY USA > [email protected] > +1 503 285 4963 > +1 707 671 1333 fax > > www.truist.com > > > The information contained in this email/document is confidential and may be > legally privileged. Access to this mail/document by anyone other than the > intended recipient(s) is unauthorized. If you are not an intended recipient, > any disclosure, copying, distribution, or any action taken or omitted to be > taken in reliance to it, is prohibited. > > > ---------- Forwarded message ---------- > From: Jack Schmidt <[email protected]> > Date: Thu, Apr 30, 2009 at 4:03 PM > Subject: Re: [AOLSERVER] TLS 1.6 and Aolserver > To: Jade Rubick <[email protected]> > Cc: tech <[email protected]> > > > I just tried it by recompiling openssl with threads as compiler option and > it produces the same problem. Maybe there's another way of making openssl > thread safe. Not sure as of the moment. > > 2009/5/1 Jack Schmidt <[email protected]> >> >> It's certainly a possibility. Since I'm also trying to debianize openssl >> from Gutsy with debug symbols, we can easily slip in a threaded build. >> >> 2009/5/1 Jade Rubick <[email protected]> >>> >>> Maybe we didn't compile openssl to be threadsafe? >>> J >>> >>> Jade Rubick >>> >>> Director of Development >>> >>> TRUiST >>> >>> 120 Wall Street, 4th Floor >>> >>> New York, NY 10005 USA >>> >>> [email protected] >>> +1 503 285 4963 >>> >>> +1 707 671 1333 fax >>> >>> www.truist.com >>> >>> The information contained in this email/document is confidential and may >>> be legally privileged. Access to this email/document by anyone other than >>> the intended recipient(s) is unauthorized. If you are not an intended >>> recipient, any disclosure, copying, distribution, or any action taken or >>> omitted to be taken in reliance to it, is prohibited. >>> Begin forwarded message: >>> >>> From: Andrew Steets <[email protected]> >>> Date: April 29, 2009 6:16:14 PM PDT >>> To: [email protected] >>> Subject: Re: [AOLSERVER] TLS 1.6 and Aolserver >>> Reply-To: AOLserver Discussion <[email protected]> >>> Hello, >>> >>> We don't use this TLS package at Wayport, but I have seen similar >>> errors with OpenSSL before in other applications. I pulled the TLS >>> code and glanced through it. It doesn't look like you have registered >>> the locking callbacks for openssl, which means any openssl calls are >>> not thread safe. That's going to be a problem inside aolserver :-) >>> >>> Check out InitOpenSSL() nsopenssl.c (in the nsopenssl module). It >>> does all the basic stuff you need to get OpenSSL running in a >>> thread-safe manor. >>> >>> Also: http://openssl.org/docs/crypto/threads.html >>> >>> If you 'info threads' and see other threads inside openssl crypto >>> functions this is almost certainly your problem. >>> >>> HTH. >>> >>> -Andrew >>> >>> On Wed, Apr 29, 2009 at 5:29 PM, Jade Rubick <[email protected]> wrote: >>> >>> Jeff: >>> >>> Here is a backtrace of the crash with 1.6 stable. Did you need it from >>> head? >>> >>> J >>> >>> Jade Rubick >>> >>> Director of Development >>> >>> TRUiST >>> >>> 120 Wall Street, 4th Floor >>> >>> New York, NY 10005 USA >>> >>> [email protected] >>> >>> +1 503 285 4963 >>> >>> +1 707 671 1333 fax >>> >>> www.truist.com >>> >>> The information contained in this email/document is confidential and may >>> be >>> >>> legally privileged. Access to this email/document by anyone other than >>> the >>> >>> intended recipient(s) is unauthorized. If you are not an intended >>> recipient, >>> >>> any disclosure, copying, distribution, or any action taken or omitted to >>> be >>> >>> taken in reliance to it, is prohibited. >>> >>> Begin forwarded message: >>> >>> TLS BACKTRACE FROM 1.6 stable (without disabling DH) >>> >>> Complete backtrace: >>> >>> (gdb) bt >>> >>> #0 0xffffe410 in __kernel_vsyscall () >>> >>> #1 0xb7cd4875 in raise () from /lib/tls/i686/cmov/libc.so.6 >>> >>> #2 0xb7cd6201 in abort () from /lib/tls/i686/cmov/libc.so.6 >>> >>> #3 0xb7ee7a4f in Tcl_PanicVA () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #4 0xb7ee7a77 in Tcl_Panic () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #5 0xb7ef6b4f in Ptr2Block () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #6 0xb7ef7117 in TclpFree () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #7 0xb7e9751d in Tcl_Free () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #8 0xb7f27251 in ns_free () from >>> >>> /usr/local/aolserver40r10/lib/libnsthread.so >>> >>> #9 0xb605c4aa in CRYPTO_free () from >>> /usr/lib/i686/cmov/libcrypto.so.0.9.8 >>> >>> #10 0xb60890aa in BN_clear_free () from >>> >>> /usr/lib/i686/cmov/libcrypto.so.0.9.8 >>> >>> #11 0xb60b0836 in DH_free () from /usr/lib/i686/cmov/libcrypto.so.0.9.8 >>> >>> #12 0xa1ffa1e5 in CTX_Init (statePtr=0x139ce5c0, proto=3, key=0x0, >>> cert=0x0, >>> >>> CAdir=0x0, CAfile=0x0, ciphers=0x0) at tls.c:1015 >>> >>> #13 0xa1ff9a72 in ImportObjCmd (clientData=0x0, interp=0x16403240, >>> objc=4, >>> >>> objv=0xa97f96bc) at tls.c:800 >>> >>> #14 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #15 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #16 0xb7e93635 in Tcl_EvalObjEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #17 0xb7e9a358 in Tcl_EvalObjCmd () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #18 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #19 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #20 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #21 0xb7eefd68 in TclObjInterpProc () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #22 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #23 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #24 0xb7e93635 in Tcl_EvalObjEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #25 0xb7e9a358 in Tcl_EvalObjCmd () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #26 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #27 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #28 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #29 0xb7eefd68 in TclObjInterpProc () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #30 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #31 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #32 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #33 0xb7e93539 in Tcl_EvalObjEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #34 0xb7e9fe07 in Tcl_IfObjCmd () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #35 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #36 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #37 0xb7edcccb in Tcl_FSEvalFile () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #38 0xb7ea5f16 in Tcl_SourceObjCmd () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #39 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #40 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #41 0xb7e93635 in Tcl_EvalObjEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #42 0xb7ee3cf1 in Tcl_NamespaceObjCmd () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #43 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #44 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #45 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #46 0xb7eefd68 in TclObjInterpProc () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #47 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #48 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #49 0xb7e93635 in Tcl_EvalObjEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #50 0xb7ef05bd in Tcl_UplevelObjCmd () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #51 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #52 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #53 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #54 0xb7eefd68 in TclObjInterpProc () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #55 0xb7ee13c9 in InvokeImportedCmd () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #56 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #57 0xb7ebf0db in TclExecuteByteCode () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #58 0xb7ec2dbc in TclCompEvalObj () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #59 0xb7eefd68 in TclObjInterpProc () from >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #60 0xb7e923c3 in TclEvalObjvInternal () from >>> >>> /usr/local/tcl/lib/libtcl8.4.so >>> >>> #61 0xb7e92987 in Tcl_EvalEx () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #62 0xb7e92c8c in Tcl_Eval () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #63 0xb7e92d26 in Tcl_GlobalEval () from /usr/local/tcl/lib/libtcl8.4.so >>> >>> #64 0xb7f68e83 in ProcRequest () from >>> >>> /usr/local/aolserver40r10/lib/libnsd.so >>> >>> #65 0xb7f5523b in Ns_ConnRunRequest () >>> >>> from /usr/local/aolserver40r10/lib/libnsd.so >>> >>> #66 0xb7f569fc in NsConnThread () from >>> >>> /usr/local/aolserver40r10/lib/libnsd.so >>> >>> #67 0xb7f2848f in NsThreadMain () >>> >>> from /usr/local/aolserver40r10/lib/libnsthread.so >>> >>> #68 0xb7f29b6d in ThreadMain () >>> >>> from /usr/local/aolserver40r10/lib/libnsthread.so >>> >>> #69 0xb7e3f46b in start_thread () from /lib/tls/i686/cmov/libpthread.so.0 >>> >>> #70 0xb7d7d6de in clone () from /lib/tls/i686/cmov/libc.so.6 >>> >>> Offending frame in stack: >>> >>> #12 0xa1ffa1e5 in CTX_Init (statePtr=0x139ce5c0, proto=3, key=0x0, >>> cert=0x0, >>> >>> CAdir=0x0, CAfile=0x0, ciphers=0x0) at tls.c:1015 >>> >>> 1015 DH_free(dh); >>> >>> Shared Libraries loaded at time of crash: >>> >>> (gdb) info sharedlibrary >>> >>> From To Syms Read Shared Object Library >>> >>> 0xb7f847f0 0xb7f991af Yes /lib/ld-linux.so.2 >>> >>> 0xb7f3f4f0 0xb7f75144 Yes >>> /usr/local/aolserver40r10/lib/libnsd.so >>> >>> 0xb7f27150 0xb7f29e94 Yes >>> >>> /usr/local/aolserver40r10/lib/libnsthread.so >>> >>> 0xb7e853b0 0xb7f0c944 Yes /usr/local/tcl/lib/libtcl8.4.so >>> >>> 0xb7e52a70 0xb7e53a74 Yes /lib/tls/i686/cmov/libdl.so.2 >>> >>> 0xb7e3e250 0xb7e49224 Yes /lib/tls/i686/cmov/libpthread.so.0 >>> >>> 0xb7e18460 0xb7e323f4 Yes /lib/tls/i686/cmov/libm.so.6 >>> >>> 0xb7e01910 0xb7e0f104 Yes /usr/lib/libz.so.1 >>> >>> 0xb7df5970 0xb7dfce64 Yes /lib/libgcc_s.so.1 >>> >>> 0xb7cbfe30 0xb7dbcb24 Yes /lib/tls/i686/cmov/libc.so.6 >>> >>> 0xb7c9f920 0xb7ca6364 Yes /lib/tls/i686/cmov/libnss_files.so.2 >>> >>> 0xb7c95e70 0xb7c9af44 Yes /lib/tls/i686/cmov/libnss_compat.so.2 >>> >>> 0xb7c80170 0xb7c8df14 Yes /lib/tls/i686/cmov/libnsl.so.1 >>> >>> 0xb7c74920 0xb7c7a654 Yes /lib/tls/i686/cmov/libnss_nis.so.2 >>> >>> 0xb73263e0 0xb7326630 Yes >>> /usr/local/aolserver40r10/bin/nssock.so >>> >>> 0xb7e6c050 0xb7e6d534 Yes >>> /usr/local/aolserver40r10/bin/nslog.so >>> >>> 0xb7e69370 0xb7e693d0 Yes /usr/local/aolserver40r10/bin/nsdb.so >>> >>> 0xb7e61f90 0xb7e669d4 Yes >>> /usr/local/aolserver40r10/lib/libnsdb.so >>> >>> 0xb7219940 0xb7221ce4 Yes /usr/local/aolserver/bin/nsoracle.so >>> >>> 0xb65d6b50 0xb70e1894 Yes >>> >>> /opt/oracle/product/10gR2/lib/libclntsh.so.10.1 >>> >>> 0xb63076c0 0xb646e5a4 Yes >>> >>> /opt/oracle/product/10gR2/lib/libnnz10.so >>> >>> 0xb61c1840 0xb61c39a4 Yes >>> /usr/local/aolserver40r10/bin/nscgi.so >>> >>> 0xb7e5c150 0xb7e5dea4 Yes >>> /usr/local/aolserver40r10/bin/nscache.so >>> >>> 0xb7e58a10 0xb7e59514 Yes >>> /usr/local/aolserver40r10/bin/nszlib.so >>> >>> 0xb61bd480 0xb61be844 Yes /usr/local/aolserver/bin/nsopenssl.so >>> >>> 0xb616e020 0xb6199b04 Yes /usr/lib/i686/cmov/libssl.so.0.9.8 >>> >>> 0xb605b140 0xb612be64 Yes /usr/lib/i686/cmov/libcrypto.so.0.9.8 >>> >>> 0xb6016b30 0xb601c734 Yes >>> >>> /usr/local/aolserver40r10//lib/libnsopenssl.so >>> >>> 0xb5fe3df0 0xb600a634 Yes >>> >>> /usr/local/aolserver40r10/lib/xotcl1.5.6/libxotcl1.5.6.so >>> >>> 0xb5ef4750 0xb5f51454 Yes >>> >>> /usr/local/tcl/lib/tdom0.8.0/libtdom0.8.0.so >>> >>> 0xa1ff7300 0xa1ffdba4 Yes >>> /usr/local/tcl/lib/tls1.6/libtls1.6.so >>> >>> 0xb5fdf770 0xb5fe03a4 Yes /lib/libnss_mdns4_minimal.so.2 >>> >>> 0xb5fd9b90 0xb5fdc714 Yes /lib/tls/i686/cmov/libnss_dns.so.2 >>> >>> 0xb5fc8110 0xb5fd3314 Yes /lib/tls/i686/cmov/libresolv.so.2 >>> >>> >>> -- >>> >>> "A scrum a day keeps the pigs at bay" >>> >>> >>> -- >>> >>> AOLserver - http://www.aolserver.com/ >>> >>> To Remove yourself from this list, simply send an email to >>> >>> <[email protected]> with the >>> >>> body of "SIGNOFF AOLSERVER" in the email message. You can leave the >>> Subject: >>> >>> field of your email blank. >>> >>> >>> >>> -- >>> AOLserver - http://www.aolserver.com/ >>> >>> To Remove yourself from this list, simply send an email to >>> <[email protected]> with the >>> body of "SIGNOFF AOLSERVER" in the email message. You can leave the >>> Subject: field of your email blank. >>> >> >> >> >> -- >> "A scrum a day keeps the pigs at bay" > > > > -- > "A scrum a day keeps the pigs at bay" > > > -- > AOLserver - http://www.aolserver.com/ > > To Remove yourself from this list, simply send an email to > <[email protected]> with the > body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: > field of your email blank. > -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[email protected]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
