Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
As a quick update. I'm using TCL threads to do this. The CRYPTO_* functions are, unsurprisingly used as defaults for openssl. What was happening in my code was that if I set CRYPTO_lock to be the lock callback function, it would end up in an infinite loop. The problem so far has been that as far as I understand, OpenSSL specifies that the functions must support n mutex locks where n is extracted from CRYPTO_num_locks(). I'll see if it is possible to do this with TCL_DECLARE_MUTEX. I appreciate all the info thus far. :) On May 5, 1:06 pm, Sep Ng thejackschm...@gmail.com wrote: Thanks for the link Jeff. I'm still trying to figure out if it's possible to use CRYPTO_* for the mutex though I agree 100% with you that if it had been the case, they wouldn't have a need for explicitly defining the function. As of now, using the CRYPTO_lock functions seem to yield some sort of deadlock where all the threads stop at some point until the SIGSEGV signal is emitted. The backtrace looks funny so I'll look at TCL threads after I exhaust this option. On May 5, 11:15 am, Jeff Hobbs je...@activestate.com wrote: Taking a quick look, that does appear to be perfectly matched to the callback that they want. Of course, if that is the case I wonder why they say this must be set, rather than making it optional. Otherwise you have Tcl_MutexLock and the related functions mentioned at: http://www.tcl.tk/man/tcl8.5/TclLib/Thread.htm For CRYPTO_set_id_callback, it looks like Tcl_GetCurrentThread is the right callback. On 4-May-09, at 7:43 PM, Sep Ng wrote: I'm working on this on behalf of Jade and I'd like to get some info on adding thread safe code on TLS. As noted by Andrew (huge thanks!), TLS does not set the CRYPTO_set_locking_callback and CRYPTO_set_id_callback callback functions which would be required to have TLS thread safe code. I was working on possibly using pthreads for mutex, but came across that OpenSSL does indeed come with Thread support. I would suppose it would be as straight forward as using the mutex designed by OpenSSL (e.g. CRYPTO_lock, etc.). I'd like to know if I'm heading to the right direction with this. I've begun reading up on OpenSSL's threads API but would appreciate any help. Thank you very much! On May 5, 6:45 am, Jade Rubick jrub...@truist.com wrote: Thank you, Andrew. We'll look into that. J Jade Rubick Director of Development TRUiST 120 Wall Street, 4th Floor New York, NY USA jrub...@truist.com +1 503 285 4963 +1 707 671 1333 fax www.truist.com The information contained in this email/document is confidential and may be legally privileged. Access to this mail/document by anyone other than the intended recipient(s) is unauthorized. If you are not an intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance to it, is prohibited. On Fri, May 1, 2009 at 12:42 PM, Andrew Steets ste...@gmail.com wrote: It's not a matter of compiling OpenSSL to be thread safe. Someone needs to update the TLS C code to call the right OpenSSL API functions on module initialization. In it's current state I don't see how the TLS module can safely call OpenSSL from a threaded context. From the Openssl docs: http://openssl.org/docs/crypto/threads.html#DESCRIPTION OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set, locking_function and threadid_func. The TLS C code doesn't setup either one of those callbacks, so that's a problem. I'm not sure if that is your problem specifically but it would be a good place to start. -Andrew On Fri, May 1, 2009 at 12:59 PM, Jade Rubick jrub...@truist.com wrote: Jade Rubick Director of Development TRUiST 120 Wall Street, 4th Floor New York, NY USA jrub...@truist.com +1 503 285 4963 +1 707 671 1333 fax www.truist.com The information contained in this email/document is confidential and may be legally privileged. Access to this mail/document by anyone other than the intended recipient(s) is unauthorized. If you are not an intended recipient, any disclosure, copying, distribution, or any action taken or omitted to be taken in reliance to it, is prohibited. -- Forwarded message -- From: Jack Schmidt thejackschm...@gmail.com Date: Thu, Apr 30, 2009 at 4:03 PM Subject: Re: [AOLSERVER] TLS 1.6 and Aolserver To: Jade Rubick jrub...@truist.com Cc: tech t...@volunteersolutions.org I just tried it by recompiling openssl with threads as compiler option and it produces the same problem. Maybe there's another way of making openssl thread safe. Not sure as of the moment. 2009/5/1 Jack Schmidt thejackschm...@gmail.com It's certainly a possibility. Since I'm
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
I certainly hope I'm not spamming but this is likely to be the last
update of the day (at least on my end).
I wrote a pretty sketchy (and lots of bad programming) but I think I
*might* have gotten the mutex initialization done. Unfortunately, on
tests, it falls on a segmentation fault on exactly the same spot
(DH_free, etc.).
Here is the diff right now:
--- tls.c.back 2009-05-05 10:06:59.0 +0800
+++ tls.c 2009-05-05 15:41:16.0 +0800
@@ -130,6 +130,58 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#if defined(OPENSSL_THREADS)
+
+#include openssl/crypto.h
+
+/*
+ * This is likely to be nasty coding practices
+ * Based from /crypto/cryptlib.c of OpenSSL
+ * Also based on NSOpenSSL
+ *
+ */
+
+Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+size_t num_locks;
+
+static void CryptoThreadLockCallback(int mode, int n, const char
*file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (n = num_locks)
+{
+n = num_locks - 1;
+}
+
+if (mode CRYPTO_LOCK)
+{
+ Tcl_MutexLock(locks[n]);
+ //Tcl_MutexLock(locks);
+}
+else
+{
+ Tcl_MutexUnlock(locks[n]);
+ //Tcl_MutexUnlock(locks);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+
+#endif
+
/*
*---
@@ -1500,6 +1552,22 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+#if defined(OPENSSL_THREADS)
+
+num_locks = CRYPTO_num_locks();
+int lock = 0;
+for (lock = 0; lock num_locks; lock++)
+{
+ TCL_DECLARE_MUTEX(mutex);
+ locks[lock]=mutex;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+
+#endif
+
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
We cannot use CRYPTO_lock because CRYPTO_lock calls the function we
set with CRYPTO_set_locking_callback, so this is completely out of the
picture. I agree with Jeff that TCL threads and mutex is the right
way to handle this but I'm wondering if the code I wrote has some
incorrect implementation, which is leading to still the same crash
happening.
Minor point is that I have yet to find a place to run
Tcl_Finalize_mutex so we can unload the mutex. Should help prevent
memory leaks, I think.
I will e-mail Jade and Jeff the backtrace as I think it will only muck
up the discussion.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
On 05/05/2009 12:55 AM, Sep Ng wrote:
I certainly hope I'm not spamming but this is likely to be the last
update of the day (at least on my end).
I wrote a pretty sketchy (and lots of bad programming) but I think I
*might* have gotten the mutex initialization done. Unfortunately, on
tests, it falls on a segmentation fault on exactly the same spot
(DH_free, etc.).
Here is the diff right now:
--- tls.c.back 2009-05-05 10:06:59.0 +0800
+++ tls.c 2009-05-05 15:41:16.0 +0800
@@ -130,6 +130,58 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#if defined(OPENSSL_THREADS)
+
+#include openssl/crypto.h
+
+/*
+ * This is likely to be nasty coding practices
+ * Based from /crypto/cryptlib.c of OpenSSL
+ * Also based on NSOpenSSL
+ *
+ */
+
+Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+size_t num_locks;
+
+static void CryptoThreadLockCallback(int mode, int n, const char
*file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (n = num_locks)
+{
+n = num_locks - 1;
+}
+
+if (mode CRYPTO_LOCK)
+{
+ Tcl_MutexLock(locks[n]);
+ //Tcl_MutexLock(locks);
+}
+else
+{
+ Tcl_MutexUnlock(locks[n]);
+ //Tcl_MutexUnlock(locks);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+
+#endif
+
/*
*---
@@ -1500,6 +1552,22 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+#if defined(OPENSSL_THREADS)
+
+num_locks = CRYPTO_num_locks();
+int lock = 0;
+for (lock = 0; lock num_locks; lock++)
+{
+ TCL_DECLARE_MUTEX(mutex);
+ locks[lock]=mutex;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+
+#endif
+
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
We cannot use CRYPTO_lock because CRYPTO_lock calls the function we
set with CRYPTO_set_locking_callback, so this is completely out of the
picture. I agree with Jeff that TCL threads and mutex is the right
way to handle this but I'm wondering if the code I wrote has some
incorrect implementation, which is leading to still the same crash
happening.
Minor point is that I have yet to find a place to run
Tcl_Finalize_mutex so we can unload the mutex. Should help prevent
memory leaks, I think.
I will e-mail Jade and Jeff the backtrace as I think it will only muck
up the discussion.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
On 05/05/2009 12:55 AM, Sep Ng wrote:
I certainly hope I'm not spamming but this is likely to be the last
update of the day (at least on my end).
I wrote a pretty sketchy (and lots of bad programming) but I think I
*might* have gotten the mutex initialization done. Unfortunately, on
tests, it falls on a segmentation fault on exactly the same spot
(DH_free, etc.).
Here is the diff right now:
--- tls.c.back 2009-05-05 10:06:59.0 +0800
+++ tls.c 2009-05-05 15:41:16.0 +0800
@@ -130,6 +130,58 @@
#define sk_SSL_CIPHER_value( sk, index) (SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#if defined(OPENSSL_THREADS)
+
+#include openssl/crypto.h
+
+/*
+ * This is likely to be nasty coding practices
+ * Based from /crypto/cryptlib.c of OpenSSL
+ * Also based on NSOpenSSL
+ *
+ */
+
+Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+size_t num_locks;
+
+static void CryptoThreadLockCallback(int mode, int n, const char
*file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (n = num_locks)
+{
+n = num_locks - 1;
+}
+
+if (mode CRYPTO_LOCK)
+{
+ Tcl_MutexLock(locks[n]);
+ //Tcl_MutexLock(locks);
+}
+else
+{
+ Tcl_MutexUnlock(locks[n]);
+ //Tcl_MutexUnlock(locks);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+
+#endif
+
/*
*---
@@ -1500,6 +1552,22 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+#if defined(OPENSSL_THREADS)
+
+num_locks = CRYPTO_num_locks();
+int lock = 0;
+for (lock = 0; lock num_locks; lock++)
+{
+ TCL_DECLARE_MUTEX(mutex);
+ locks[lock]=mutex;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+
+#endif
+
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
We cannot use CRYPTO_lock because CRYPTO_lock calls the function we
set with CRYPTO_set_locking_callback, so this is completely out of the
picture. I agree with Jeff that TCL threads and mutex is the right
way to handle this but I'm wondering if the code I wrote has some
incorrect implementation, which is leading to still the same crash
happening.
Minor point is that I have yet to find a place to run
Tcl_Finalize_mutex so we can unload the mutex. Should help prevent
memory leaks, I think.
I will e-mail Jade and Jeff the backtrace as I think it will only muck
up the discussion.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
Of the presented patches, I didn't find one that seemed to actually
work, so I wrote one based on those presented. It is attached. Please
test it in your environments. I have tested that it passes the basic
tls test suite against a threaded Tcl 8.5.7 core build on Linux-x64 (and
verified that OPENSSL_THREADS was true for this install).
This patch is against tls 1.6 head.
Jeff
On 05/05/2009 3:42 PM, Sep Ng wrote:
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
? announce.txt
? tls-thread.diff
? tls-verify.diff
Index: Makefile.in
===
RCS file: /cvsroot/tls/tls/Makefile.in,v
retrieving revision 1.27
diff -u -r1.27 Makefile.in
--- Makefile.in 19 Mar 2008 22:57:03 - 1.27
+++ Makefile.in 5 May 2009 23:52:38 -
@@ -205,7 +205,7 @@
file copy [file join $(srcdir) tls.tcl] tls.tcl \
} ;\
source [file join $(srcdir) tls.tcl]; \
- set argv $(TESTFLAGS); \
+ set argv {$(TESTFLAGS)}; \
source [file join $(srcdir) tests all.tcl] | $(TCLSH)
shell: binaries libraries
Index: tls.c
===
RCS file: /cvsroot/tls/tls/tls.c,v
retrieving revision 1.30
diff -u -r1.30 tls.c
--- tls.c 19 Mar 2008 22:06:13 - 1.30
+++ tls.c 5 May 2009 23:52:38 -
@@ -130,6 +130,46 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#ifdef TCL_THREADS
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#ifdef OPENSSL_THREADS
+#include openssl/crypto.h
+
+/*
+ * Threaded operation requires locking callbacks
+ * Based from /crypto/cryptlib.c of OpenSSL and NSOpenSSL.
+ */
+
+static Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+
+static void CryptoThreadLockCallback(int mode, int n,
+ const char *file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (mode CRYPTO_LOCK) {
+ Tcl_MutexLock(locks[n]);
+} else {
+ Tcl_MutexUnlock(locks[n]);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+#endif /* OPENSSL_THREADS */
+#endif /* TCL_THREADS */
+
/*
*---
@@ -1468,6 +1508,9 @@
{
int major, minor, patchlevel, release, i;
char rnd_seed[16] = GrzSlplKqUdnnzP!;/* 16 bytes */
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+size_t num_locks;
+#endif
/*
* The original 8.2.0 stacked channel implementation (and the patch
@@ -1500,6 +1543,24 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+if (CRYPTO_set_mem_functions((void *(*)(size_t))Tcl_Alloc,
+ (void *(*)(void *, size_t))Tcl_Realloc,
+ (void(*)(void *))Tcl_Free) == 0) {
+ /* Not using Tcl's mem functions ... not critical */
+}
+
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+/* should we consider allocating mutexes? */
+num_locks = CRYPTO_num_locks();
+if (num_locks CRYPTO_NUM_LOCKS) {
+ Tcl_AppendResult(interp, crypto num locks size error, NULL);
+ return TCL_ERROR;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+#endif
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
It seems that Tcl's Tcl_Alloc, Tcl_Realloc, and Tcl_Free are defined
differently than the ones expected by CRYPTO_set_mem_functions (the
functions are expected to be of void*). I tried to put a wrapper
around it but I haven't come across much success in that. I will see
if there are other avenues to this, maybe even using the standard
malloc might do.
On May 6, 6:42 am, Sep Ng thejackschm...@gmail.com wrote:
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
On 05/05/2009 12:55 AM, Sep Ng wrote:
I certainly hope I'm not spamming but this is likely to be the last
update of the day (at least on my end).
I wrote a pretty sketchy (and lots of bad programming) but I think I
*might* have gotten the mutex initialization done. Unfortunately, on
tests, it falls on a segmentation fault on exactly the same spot
(DH_free, etc.).
Here is the diff right now:
--- tls.c.back 2009-05-05 10:06:59.0 +0800
+++ tls.c 2009-05-05 15:41:16.0 +0800
@@ -130,6 +130,58 @@
#define sk_SSL_CIPHER_value( sk, index) (SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#if defined(OPENSSL_THREADS)
+
+#include openssl/crypto.h
+
+/*
+ * This is likely to be nasty coding practices
+ * Based from /crypto/cryptlib.c of OpenSSL
+ * Also based on NSOpenSSL
+ *
+ */
+
+Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+size_t num_locks;
+
+static void CryptoThreadLockCallback(int mode, int n, const char
*file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (n = num_locks)
+{
+n = num_locks - 1;
+}
+
+if (mode CRYPTO_LOCK)
+{
+ Tcl_MutexLock(locks[n]);
+ //Tcl_MutexLock(locks);
+}
+else
+{
+ Tcl_MutexUnlock(locks[n]);
+ //Tcl_MutexUnlock(locks);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+
+#endif
+
/*
*---
@@ -1500,6 +1552,22 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+#if defined(OPENSSL_THREADS)
+
+num_locks = CRYPTO_num_locks();
+int lock = 0;
+for (lock = 0; lock num_locks; lock++)
+{
+ TCL_DECLARE_MUTEX(mutex);
+ locks[lock]=mutex;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+
+#endif
+
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
We cannot use CRYPTO_lock because CRYPTO_lock calls the function we
set with CRYPTO_set_locking_callback, so this is completely out of the
picture. I agree with Jeff that TCL threads and mutex is the right
way to handle this but I'm wondering if the code I wrote has some
incorrect implementation, which is leading to still the same crash
happening.
Minor point is that I have yet to find a place to run
Tcl_Finalize_mutex so we can unload the mutex. Should help prevent
memory leaks, I think.
I will e-mail Jade and Jeff the backtrace as I think it will only muck
up the discussion.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
I just tried your patch, Jeff. It still crashes. I'll try to get a
backtrace right now.
On May 6, 7:53 am, Jeff Hobbs je...@activestate.com wrote:
Of the presented patches, I didn't find one that seemed to actually
work, so I wrote one based on those presented. It is attached. Please
test it in your environments. I have tested that it passes the basic
tls test suite against a threaded Tcl 8.5.7 core build on Linux-x64 (and
verified that OPENSSL_THREADS was true for this install).
This patch is against tls 1.6 head.
Jeff
On 05/05/2009 3:42 PM, Sep Ng wrote:
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
[tls-thread.diff3K ]? announce.txt
? tls-thread.diff
? tls-verify.diff
Index: Makefile.in
===
RCS file: /cvsroot/tls/tls/Makefile.in,v
retrieving revision 1.27
diff -u -r1.27 Makefile.in
--- Makefile.in 19 Mar 2008 22:57:03 - 1.27
+++ Makefile.in 5 May 2009 23:52:38 -
@@ -205,7 +205,7 @@
file copy [file join $(srcdir) tls.tcl] tls.tcl \
} ;\
source [file join $(srcdir) tls.tcl]; \
- set argv $(TESTFLAGS); \
+ set argv {$(TESTFLAGS)}; \
source [file join $(srcdir) tests all.tcl] | $(TCLSH)
shell: binaries libraries
Index: tls.c
===
RCS file: /cvsroot/tls/tls/tls.c,v
retrieving revision 1.30
diff -u -r1.30 tls.c
--- tls.c 19 Mar 2008 22:06:13 - 1.30
+++ tls.c 5 May 2009 23:52:38 -
@@ -130,6 +130,46 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#ifdef TCL_THREADS
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#ifdef OPENSSL_THREADS
+#include openssl/crypto.h
+
+/*
+ * Threaded operation requires locking callbacks
+ * Based from /crypto/cryptlib.c of OpenSSL and NSOpenSSL.
+ */
+
+static Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+
+static void CryptoThreadLockCallback(int mode, int n,
+ const char *file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (mode CRYPTO_LOCK) {
+ Tcl_MutexLock(locks[n]);
+} else {
+ Tcl_MutexUnlock(locks[n]);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+#endif /* OPENSSL_THREADS */
+#endif /* TCL_THREADS */
+
/*
*---
@@ -1468,6 +1508,9 @@
{
int major, minor, patchlevel, release, i;
char rnd_seed[16] = GrzSlplKqUdnnzP!; /* 16 bytes */
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+size_t num_locks;
+#endif
/*
* The original 8.2.0 stacked channel implementation (and the patch
@@ -1500,6 +1543,24 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+if (CRYPTO_set_mem_functions((void *(*)(size_t))Tcl_Alloc,
+ (void *(*)(void *, size_t))Tcl_Realloc,
+ (void(*)(void *))Tcl_Free) == 0) {
+ /* Not using Tcl's mem functions ... not critical */
+}
+
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+/* should we consider allocating mutexes? */
+num_locks = CRYPTO_num_locks();
+if (num_locks CRYPTO_NUM_LOCKS) {
+ Tcl_AppendResult(interp, crypto num locks size error, NULL);
+ return TCL_ERROR;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+#endif
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
I've just yanked the debug. This includes the backtrace and memory frame
info and the local info for most of the frames up until #11 CTX_Init. As
before, the crash happens when DH_free is called.
2009/5/6 Jeff Hobbs je...@activestate.com
Of the presented patches, I didn't find one that seemed to actually work,
so I wrote one based on those presented. It is attached. Please test it in
your environments. I have tested that it passes the basic tls test suite
against a threaded Tcl 8.5.7 core build on Linux-x64 (and verified that
OPENSSL_THREADS was true for this install).
This patch is against tls 1.6 head.
Jeff
On 05/05/2009 3:42 PM, Sep Ng wrote:
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
? announce.txt
? tls-thread.diff
? tls-verify.diff
Index: Makefile.in
===
RCS file: /cvsroot/tls/tls/Makefile.in,v
retrieving revision 1.27
diff -u -r1.27 Makefile.in
--- Makefile.in 19 Mar 2008 22:57:03 - 1.27
+++ Makefile.in 5 May 2009 23:52:38 -
@@ -205,7 +205,7 @@
file copy [file join $(srcdir) tls.tcl] tls.tcl \
} ;\
source [file join $(srcdir) tls.tcl]; \
- set argv $(TESTFLAGS); \
+ set argv {$(TESTFLAGS)}; \
source [file join $(srcdir) tests all.tcl] | $(TCLSH)
shell: binaries libraries
Index: tls.c
===
RCS file: /cvsroot/tls/tls/tls.c,v
retrieving revision 1.30
diff -u -r1.30 tls.c
--- tls.c 19 Mar 2008 22:06:13 - 1.30
+++ tls.c 5 May 2009 23:52:38 -
@@ -130,6 +130,46 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#ifdef TCL_THREADS
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#ifdef OPENSSL_THREADS
+#include openssl/crypto.h
+
+/*
+ * Threaded operation requires locking callbacks
+ * Based from /crypto/cryptlib.c of OpenSSL and NSOpenSSL.
+ */
+
+static Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+
+static void CryptoThreadLockCallback(int mode, int n,
+ const char *file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (mode CRYPTO_LOCK) {
+ Tcl_MutexLock(locks[n]);
+} else {
+ Tcl_MutexUnlock(locks[n]);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+#endif /* OPENSSL_THREADS */
+#endif /* TCL_THREADS */
+
/*
*---
@@ -1468,6 +1508,9 @@
{
int major, minor, patchlevel, release, i;
char rnd_seed[16] = GrzSlplKqUdnnzP!;/* 16 bytes */
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+size_t num_locks;
+#endif
/*
* The original 8.2.0 stacked channel implementation (and the patch
@@ -1500,6 +1543,24 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+if (CRYPTO_set_mem_functions((void *(*)(size_t))Tcl_Alloc,
+ (void *(*)(void *, size_t))Tcl_Realloc,
+ (void(*)(void *))Tcl_Free) == 0) {
+ /* Not using Tcl's mem functions ... not critical */
+}
+
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+/* should we consider allocating mutexes? */
+num_locks = CRYPTO_num_locks();
+if (num_locks CRYPTO_NUM_LOCKS) {
+ Tcl_AppendResult(interp, crypto num locks size error, NULL);
+ return TCL_ERROR;
+}
+
+CRYPTO_set_locking_callback(CryptoThreadLockCallback);
+CRYPTO_set_id_callback(CryptoThreadIdCallback);
+#endif
+
if (SSL_library_init() != 1) {
Tcl_AppendResult(interp, could not initialize SSL library, NULL);
return TCL_ERROR;
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
A
Re: [AOLSERVER] Fwd: [AOLSERVER] TLS 1.6 and Aolserver
Hi Jeff,
I took a closer look at the patch you posted. It seems that the
CRYPTO_set_mem_functions is not succeeding. The default memory
functions that CRYPTO uses are malloc, realloc, and free but from the
back trace, it looks like ns_malloc, ns_realloc and ns_free are the
ones being used for some reason. I think I'm running out of ideas
here. It's unclear why CRYPTO_set_mem_function would return 0 instead
of 1, unless it's some bug in my OpenSSL package in Ubuntu.
On May 6, 8:42 am, Jack Schmidt thejackschm...@gmail.com wrote:
I've just yanked the debug. This includes the backtrace and memory frame
info and the local info for most of the frames up until #11 CTX_Init. As
before, the crash happens when DH_free is called.
2009/5/6 Jeff Hobbs je...@activestate.com
Of the presented patches, I didn't find one that seemed to actually work,
so I wrote one based on those presented. It is attached. Please test it in
your environments. I have tested that it passes the basic tls test suite
against a threaded Tcl 8.5.7 core build on Linux-x64 (and verified that
OPENSSL_THREADS was true for this install).
This patch is against tls 1.6 head.
Jeff
On 05/05/2009 3:42 PM, Sep Ng wrote:
I'll try it. I didn't give it much thought at first but looking at it
again, I think it might prevent the long string of ns_free and other
calls to free memory after DH_free.
On May 6, 3:43 am, Jeff Hobbs je...@activestate.com wrote:
Just starting to look at this, but from the nsopenssl.c I saw another
interesting function not used by TLS:
if (CRYPTO_set_mem_functions(ns_malloc, ns_realloc, ns_free) == 0) ...
We could do the same and point to Tcl_Alloc, Tcl_Realloc and Tcl_Free.
I'm not sure they are necessary, and CRYPTO_set_mem_debug_functions
isn't used, but it might help debug.
--
AOLserver -http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
? announce.txt
? tls-thread.diff
? tls-verify.diff
Index: Makefile.in
===
RCS file: /cvsroot/tls/tls/Makefile.in,v
retrieving revision 1.27
diff -u -r1.27 Makefile.in
--- Makefile.in 19 Mar 2008 22:57:03 - 1.27
+++ Makefile.in 5 May 2009 23:52:38 -
@@ -205,7 +205,7 @@
file copy [file join $(srcdir) tls.tcl] tls.tcl \
} ;\
source [file join $(srcdir) tls.tcl]; \
- set argv $(TESTFLAGS); \
+ set argv {$(TESTFLAGS)}; \
source [file join $(srcdir) tests all.tcl] | $(TCLSH)
shell: binaries libraries
Index: tls.c
===
RCS file: /cvsroot/tls/tls/tls.c,v
retrieving revision 1.30
diff -u -r1.30 tls.c
--- tls.c 19 Mar 2008 22:06:13 - 1.30
+++ tls.c 5 May 2009 23:52:38 -
@@ -130,6 +130,46 @@
#define sk_SSL_CIPHER_value( sk, index)(SSL_CIPHER*)sk_value((sk),
(index))
#endif
+/*
+ * Thread-Safe TLS Code
+ */
+
+#ifdef TCL_THREADS
+#define OPENSSL_THREAD_DEFINES
+#include openssl/opensslconf.h
+
+#ifdef OPENSSL_THREADS
+#include openssl/crypto.h
+
+/*
+ * Threaded operation requires locking callbacks
+ * Based from /crypto/cryptlib.c of OpenSSL and NSOpenSSL.
+ */
+
+static Tcl_Mutex locks[CRYPTO_NUM_LOCKS];
+
+static void CryptoThreadLockCallback(int mode, int n,
+ const char *file, int line);
+static unsigned long CryptoThreadIdCallback(void);
+
+static void
+CryptoThreadLockCallback(int mode, int n, const char *file, int line)
+{
+if (mode CRYPTO_LOCK) {
+ Tcl_MutexLock(locks[n]);
+} else {
+ Tcl_MutexUnlock(locks[n]);
+}
+}
+
+static unsigned long
+CryptoThreadIdCallback(void)
+{
+return (unsigned long) Tcl_GetCurrentThread();
+}
+#endif /* OPENSSL_THREADS */
+#endif /* TCL_THREADS */
+
/*
*---
@@ -1468,6 +1508,9 @@
{
int major, minor, patchlevel, release, i;
char rnd_seed[16] = GrzSlplKqUdnnzP!;/* 16 bytes */
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+size_t num_locks;
+#endif
/*
* The original 8.2.0 stacked channel implementation (and the patch
@@ -1500,6 +1543,24 @@
channelTypeVersion = TLS_CHANNEL_VERSION_1;
}
+if (CRYPTO_set_mem_functions((void *(*)(size_t))Tcl_Alloc,
+ (void *(*)(void *, size_t))Tcl_Realloc,
+ (void(*)(void *))Tcl_Free) == 0) {
+ /* Not using Tcl's mem functions ... not critical */
+}
+
+#if defined(OPENSSL_THREADS) defined(TCL_THREADS)
+/* should we consider allocating mutexes? */
+num_locks = CRYPTO_num_locks();
+if
