Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Klaus Wagner wrote: > On Thu, 2006-12-07 at 18:41 +, Darryl Miles wrote: >>> Maybe there is some (small) re-design of the Apache code needed? >> Agreed, something needs to be added. I'm saying there is no need to >> make it specific to OpenSSL. Serializing the initialization can be made >> generic such that these objectives are met: > > why not move this issue to libapr/libarp-util why not move this discussion to [EMAIL PROTECTED] There -is- an openssl wrapper now in apr (1.3-future release) that could fit the bill. This said, it won't be possible to accomplish the myriad features that mod_ssl exposes through a straightforward tls wrapper. We had one, mod_tls, that was dirt-simple-stupid and just-worked. It was also orphaned, people want more out of their crypto layer.
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
William A. Rowe, Jr. wrote: Darryl Miles wrote: Your thinking is correct there is a problem. Those OpenSSL functions are not documented in my man page but exist in the library. Yes there is a read-test-write race window by using those APIs alone. Nope. This is set when the server process is running in single process, single thread mode, long before the server 'opens up' and spawns off it's worker threads. Understood on the single thread situation. But what about module initialization order guaranteed and possible future modules which may also use OpenSSL ? The goal is to ensure the CRYPTO_() init functions are called, and called exactly once before openssl first use only when openssl is being used. * mod_core (no mod_ssl, no mod_frank, no need do anything no openssl users) * mod_core + mod_ssl (no mod_frank) * mod_core + mod_frank (no mod_ssl) * mod_core + mod_ssl + mod_frank * mod_core + mod_frank + mod_ssl (opposite initialization order) There may not be worker threads and apache-http may not support dynamic runtime loading / unloading of modules (at this time) but its possible to deal with all those concerns cleanly. What I DO agree with is that these callbacks should be locked in much earlier than post_config. I'm happy to see these callbacks locked in at the time we register the module itself. Is the module registration order the same for a given canonical configuration (i.e. its not subject to httpd.conf config LoadModule directive ordering). Also can one module can ask the core what other modules are loaded and it can ask them if they too are users of OpenSSL. I.e. whatever system to address this problem is used would need to deal with the possible existence of a future mod_frank_v2 (that does not exist today but may do in the future) in a way that mod_ssl does not need to be recompiled or upgraded to know about mod_frank_v2 when the time comes to release mod_frank_v2. Darryl
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Thu, 2006-12-07 at 18:41 +, Darryl Miles wrote: > > Maybe there is some (small) re-design of the Apache code needed? > > Agreed, something needs to be added. I'm saying there is no need to > make it specific to OpenSSL. Serializing the initialization can be made > generic such that these objectives are met: why not move this issue to libapr/libarp-util Through abstraction of OpenSSL (or whatever ssl implementation) all threading issues/race conditions etc. could be addressed and handled without any changes of mod_core (sure mod_ssl must be changed) After that mod_foo could use this abstraction instead of using OpenSSL. I know that this is a big peace of work and it doesn't solve the general lib_thread_save_intialisation_issue but it could be also of great use for any dev (beside httpd). regards klaus
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Frank wrote: > Joe Orton wrote: >> Yes, CRYPTO_get_locking_callback/CRYPTO_get_id_callback. >> [...] > > I already know that this functions exists. But what if my module gets > inited before mod_ssl, which doesn't use the get-functions to determine > that something is already there? I was in the hope to see a clean > general purpose solution. :-) > > After this long and informative discussion I really think there is need > for a ssl_thread_init_if_not_already_done inside Apache. (Besides the > correction/removal of OpenSSL's stupid global locking mechanism.) > > Maybe there is some (small) re-design of the Apache code needed? Honestly? I'm not happy with the thought of 'entrusting' this to another third party. If we could trust such choices, then the OpenSSL project wouldn't leave it for us to implement as a callback hook :) Unless your module spawns threads itself, before the post_config phase is complete, there is no need to call these if mod_ssl is present. Darryl Miles wrote: > > Your thinking is correct there is a problem. Those OpenSSL functions > are not documented in my man page but exist in the library. Yes there > is a read-test-write race window by using those APIs alone. Nope. This is set when the server process is running in single process, single thread mode, long before the server 'opens up' and spawns off it's worker threads. What I DO agree with is that these callbacks should be locked in much earlier than post_config. I'm happy to see these callbacks locked in at the time we register the module itself.
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Frank wrote:
Joe Orton wrote:
On Wed, Dec 06, 2006 at 06:20:55PM +, Darryl Miles wrote:
[...]
Is there an API to get the current value ?
Yes, CRYPTO_get_locking_callback/CRYPTO_get_id_callback.
[...]
I already know that this functions exists. But what if my module gets
inited before mod_ssl, which doesn't use the get-functions to determine
that something is already there? I was in the hope to see a clean
general purpose solution. :-)
Your thinking is correct there is a problem. Those OpenSSL functions
are not documented in my man page but exist in the library. Yes there
is a read-test-write race window by using those APIs alone.
After this long and informative discussion I really think there is need
for a ssl_thread_init_if_not_already_done inside Apache. (Besides the
correction/removal of OpenSSL's stupid global locking mechanism.)
I disagree I should not need to compile apache mod_core in a way that
has to understand the quirks of OpenSSL. So adding a function
ssl_thread_init_if_not_already_done() to apache core is too specific to
that one problem.
I'm saying its possible to solve a bunch of problems like this in a more
generic way. This helps you and the next man.
Maybe there is some (small) re-design of the Apache code needed?
Agreed, something needs to be added. I'm saying there is no need to
make it specific to OpenSSL. Serializing the initialization can be made
generic such that these objectives are met:
* When building Apache mod_core with minimal functionality it is not
necessary to build it with special API function to handle OpenSSL. So
we get the co-ordination mechanism by default.
* I can build my mod_ssl or mod_frank at any point later without having
to rebuild apache mod_core as well.
* Can provide a generic mechanism which can be used to solve the same
problem where a common subordinate library is linked with by a module
but is not require by mod_core.
* The overhead of adding the handful of new functions to apache mod_core
should low. Subjective but...
* Because the solution is generic it can be reused for other purposes
where one module has to co-ordinate with another module at runtime over
the use of a shared resource, but due to the modular nature of apache
its impossible to tell if/when/what other modules need to co-ordinate
their behavior until runtime.
* Only those modules that use OpenSSL directly need to be linked against
OpenSSL.
* Only those modules that link with OpenSSL need openssl around at build
time. So the helper functions below would be repeated within each
mod_ that uses OpenSSL.
Those a rough concerns to address.
All Apache should provide is an accessible keyed namespace to attach
arbitrary pieces of memory where the attachment functions provides some
thread-safe guarantees. Maybe there is already a mechanism to achieve
most of that within apache.
All these functions must be thread-safe with respect to themselves. The
key itself could be a string or an integer. Or even both if you wanted
to abstract the key, like X11 Xwindows "atoms" work, you lookup/create
your key independently and get back a reference which is usually a
cardinal integer starting at 1 and then use this api with that (not the
key directly).
* void *get(key) this gets the (void *) key value.
* void *attach(key, newdata) this overrides the current key value with
an optional exchange (by returning the old value).
* void *attach_smart_exchange(key, newdata) this does an exchange but
will only exchange a NULL for non-NULL and non-NULL for NULL. This is
used to set without overwriting. It always returns the pre-existing
value by checking that value in the context of how you called it you can
work out if you were successful or not.
Sample hypothetical usage in self-documenting non compilable code:
struct openssl_init_struct {
long magic; /* do versioning here */
int boolean_flag;
mutex_t mutex;
} *ois;
ois = calloc_from_global_pool(1, sizeof(openssl_init_struct));
ois->magic = OPENSSL_INIT_STRUCT_V1; /* magic constant */
ois->boolean_flag = FALSE;
mutex_init(&ois->mutex);
/* as first use we lock before attachment */
mutex_lock(&ois->mutex);
const char *label_key = "mylabel_openssl_initialization_serializer"; /*
could be integer based key system */
/* This ap_newapi_attach_smart_exchange function is a thread-safe atomic
attachment of "mydata" to the global namespace, it will only attach if
the current value for the key does not exist or is NULL already, which
are essentially treated identically (does not exist vs NULL value) */
void *retval =
apache_core_handle->ap_newapi_attach_smart_exchange(label_key, ois);
if(retval == NULL) {
/* we won the attachment, we already locked it and we own that lock,
fall thru to code below */
} else {
free_from_global_pool(ois); /* FIXME: destruct mutex etc.. properly
*/
/* we lost, go get the winner, you
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Joe Orton wrote: On Wed, Dec 06, 2006 at 06:20:55PM +, Darryl Miles wrote: [...] Is there an API to get the current value ? Yes, CRYPTO_get_locking_callback/CRYPTO_get_id_callback. [...] I already know that this functions exists. But what if my module gets inited before mod_ssl, which doesn't use the get-functions to determine that something is already there? I was in the hope to see a clean general purpose solution. :-) After this long and informative discussion I really think there is need for a ssl_thread_init_if_not_already_done inside Apache. (Besides the correction/removal of OpenSSL's stupid global locking mechanism.) Maybe there is some (small) re-design of the Apache code needed? But as stated before in the discussion: I will also ask the OpenSSL-mailinglist for this. Thank you for all your comments, Frank P.S.: Someone suggested in a personal mail to use the real-thread-safe http://www.geodyssey.com/cryptography/cryptography.html! (He saw that I only want to encrypt something using Blowfish.) This is an option indeed! But as a programmer I can't believe that I have to use a second Blowfish-library in Apache while there is already one available! :-)
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Darryl Miles wrote: > Frank wrote: >> William A. Rowe, Jr. wrote: >>> Nick Kew wrote: >>> [...] >>> An SSL_CTX can't be cross-threaded. If the scope of use of that CTX is >>> restricted to one thread at a time, then yes, OpenSSL has been >>> threadsafe >>> for a very very long time. >> >> You mean if I were able to create one SSL_CTX for every thread then I >> do not have to use the both thread-safe-maker callbacks? YOU don't have to set it because they are one time things, and mod_ssl establishes them for you running in a threaded MPM such as worker, or winnt. You may dig your fingers into the SSL_CTX apache uses, or create your own. If you f with the callbacks, you will blow up apache. Let mod_ssl+the MPM handle that please. > I dont think this is true. But correct my understanding too if I am > wrong. Cross-threaded might confuse someone into thinking there maybe > some "apartment threading rules" to obey, there isn't. > > "An SSL *" can't have a method invoked on the same instance at the same > time. So long as you serialize your method calls (SSL_() family) to > that same instance; any thread can call that method. It is unusual to > need to do so. > > But "SSL_CTX *" is the template context specifically designed to be > shared and used across multiple-threads if needs be, providing you make > correct use of the 'CRYPTO_set_locking_callback' and > 'CRYPTO_set_id_callback' and friends as part of your application > initialization. This allows for (amongst other things) the obviously > parallel usage of SSL_new(SSL_CTX *) when creating new connections. Good summary. I believe I misspoke, the individual SSL_XXX objects aren't thread safe (instead they are fast) but the overall SSL_CTX object is. > Maybe the openssl-users list would be a better place for assistance. Agreed
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Wed, Dec 06, 2006 at 06:20:55PM +, Darryl Miles wrote: > Joe Orton wrote: > >What I do with OpenSSL in neon is to check that the existing callback is > >NULL before registering a new callback; and likewise to check that the > >ID callback is the one neon previously registered before un-registering > >it later. If everybody did that it would be relatively safe. > > Is there an API to get the current value ? Yes, CRYPTO_get_locking_callback/CRYPTO_get_id_callback. > It shouldn't be too hard to attach a piece of memory to a keyed > namespace which the core module maintains. 3 primitives should do the > trick. data = get(key), set(key, data), set_if_null(key, data) where > the set_if_null() operation is especially thread-safe. Well, you could do this with pool userdata already but I'm not sure there's much point, the co-operation problem still exists with third-party libraries (e.g. PHP + curl). I have no idea if libraries using OpenSSL via APR-util mess with this stuff either; they could well in the future if not already. (OpenLDAP, Postgres, ...?) joe
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Joe Orton wrote: What I do with OpenSSL in neon is to check that the existing callback is NULL before registering a new callback; and likewise to check that the ID callback is the one neon previously registered before un-registering it later. If everybody did that it would be relatively safe. Is there an API to get the current value ? It shouldn't be too hard to attach a piece of memory to a keyed namespace which the core module maintains. 3 primitives should do the trick. data = get(key), set(key, data), set_if_null(key, data) where the set_if_null() operation is especially thread-safe. This would be little impact/bloat on core module, but be user extendable to allow other such things. Someone just needs to maintain an assigned key value list. A notice board for modules to talk to each other. The OpenSSL guys have actually obviated the ID callback for some future release, it was entirely unportable because of the cast-to-long issue anyway; but the locking callback remains. (void *) in the latest releases AFAIK. This is to address this very concern. Darryl
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Frank wrote: EVP_CIPHER_CTX ctx; EVP_CIPHER_CTX_init (& ctx); EVP_EncryptInit (& ctx, EVP_bf_cbc (), key, iv); EVP_EncryptUpdate (& ctx, outbuf, & olen, inbuff, n); EVP_EncryptFinal (& ctx, outbuf + olen, & tlen); Because 'EVP_CIPHER_CTX_init' is 'slow', I want to call it once! (Yes! I can call it for every request and then (I think) I am on the safe side, but I do not want this because there are MANY requests!) So my code has to be thread safe, as Apache might be compiled with thread support! To make it thread safe http://www.openssl.org/docs/crypto/threads.html told me: These functions a generally thread safe AFAIK so long as you are not using an engine / cyptocard or something shared. If its just a basic memory in memory out transform it should be pretty clean of locking. Its only stuff like SSL session cache and SSL_CTX and registrations schemes (cipher registration, digest registration, etc...) which use locking as they are shared concurrently. But I understand your point. Its a necessity because the API contract says that is what you must do. "OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set." This means the two functions 'CRYPTO_set_locking_callback' and 'CRYPTO_set_id_callback'! These two functions are being called from mod_ssl by the ssl_init_Module-function (via ssl_util_thread_setup, which creates some thread mutexes and calls the both functions) without testing whether they have already being called or not. My question is: How does this interfere with my module? How can I ensure that only one of us (mod_ssl or my module) is calling these both functions? I cannot believe that there is no problem when my module creates some thread mutexes and mod_ssl does it too... Of course it is necessary to arbitrate the load and unloading of the OpenSSL libraries. Where loading really means the initial configuration that OpenSSL requires to initialize itself before it achieves maximal thread-safety. This is true of any programming paradigm. If something it thread-unsafe until you configure it correctly to be thread-safe, you have to serialize all usage upto the point it becomes safe. This includes calling functions to initialize the safety mechanism. You also can't go and change the locking policy while there maybe one or more active users of the old/existing policy. From a practical standpoint its impossible to tell how many users there are once the first user gets stuck into application work. So you are correct, you can't blindly can go overwriting the policy with CRYPTO_set_locking_callback() to point to your mutexes. > P.S.: I still think there is need for a test routine like > 'ssl_is_thread_safe_maker_on()'. Agreed. The problem is reduced to serializing the setting of the locking policy, which is reduced to the apache http server framework providing a non-performance critical locking mechanism and a boolean flag. Then each mod_ user locks that external lock, checks the external flag, if flag not set, configure locking policy in openssl, set flag and unlock. The apache framework needs to be the arbitrator here by holding the lock and flag in a namespace accessible by all modules. It does not matter which application wins to set the policy, just so long as a policy it setup. But it would be helpful if both mod_ssl and mod_frank would agree on the same policy (OpenSSL kind of gives you a few options here) Darryl
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Wed, Dec 06, 2006 at 09:15:15AM -0800, Paul Querna wrote: > Justin Erenkrantz wrote: > > On 12/6/06, Frank <[EMAIL PROTECTED]> wrote: > >> My question is: How does this interfere with my module? How can I ensure > >> that only one of us (mod_ssl or my module) is calling these both > >> functions? I cannot believe that there is no problem when my module > > > > You just can't. I wish there was a better answer but there isn't, > > AFAIK. You might get lucky, but yah you're going to run into > > nastiness with OpenSSL's global contexts. =( -- justin > > There is always GnuTLS (or other SSL libraries) which don't have the > global context problems: > http://www.gnu.org/software/gnutls/ It kind of does still though the reliance on the horrible global error stack is not there: http://www.gnu.org/software/gnutls/manual/html_node/Multi_002dthreaded-applications.html What I do with OpenSSL in neon is to check that the existing callback is NULL before registering a new callback; and likewise to check that the ID callback is the one neon previously registered before un-registering it later. If everybody did that it would be relatively safe. The OpenSSL guys have actually obviated the ID callback for some future release, it was entirely unportable because of the cast-to-long issue anyway; but the locking callback remains. joe
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Justin Erenkrantz wrote: > On 12/6/06, Frank <[EMAIL PROTECTED]> wrote: >> My question is: How does this interfere with my module? How can I ensure >> that only one of us (mod_ssl or my module) is calling these both >> functions? I cannot believe that there is no problem when my module > > You just can't. I wish there was a better answer but there isn't, > AFAIK. You might get lucky, but yah you're going to run into > nastiness with OpenSSL's global contexts. =( -- justin There is always GnuTLS (or other SSL libraries) which don't have the global context problems: http://www.gnu.org/software/gnutls/
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On 12/6/06, Frank <[EMAIL PROTECTED]> wrote: My question is: How does this interfere with my module? How can I ensure that only one of us (mod_ssl or my module) is calling these both functions? I cannot believe that there is no problem when my module You just can't. I wish there was a better answer but there isn't, AFAIK. You might get lucky, but yah you're going to run into nastiness with OpenSSL's global contexts. =( -- justin
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Darryl Miles wrote: Nick Kew wrote: Unless OpenSSL nomenclature is rather confusing here, an SSL_CTX sounds like the kind of thing you would instantiate per-connection or per-request. Does your module act on a request or a connection? Maybe a bit of background reading and examination of reference implementations would be a better help for you right now. [...] I hope that comment wasn't for me?! There might be a misunderstanding about my original posting (or we are drifting off): Inside the OpenSSL library is more than SSL. :-) I want to use OpenSSL's crypto library (for encryption). Something like: EVP_CIPHER_CTX ctx; EVP_CIPHER_CTX_init (& ctx); EVP_EncryptInit (& ctx, EVP_bf_cbc (), key, iv); EVP_EncryptUpdate (& ctx, outbuf, & olen, inbuff, n); EVP_EncryptFinal (& ctx, outbuf + olen, & tlen); Because 'EVP_CIPHER_CTX_init' is 'slow', I want to call it once! (Yes! I can call it for every request and then (I think) I am on the safe side, but I do not want this because there are MANY requests!) So my code has to be thread safe, as Apache might be compiled with thread support! To make it thread safe http://www.openssl.org/docs/crypto/threads.html told me: "OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set." This means the two functions 'CRYPTO_set_locking_callback' and 'CRYPTO_set_id_callback'! These two functions are being called from mod_ssl by the ssl_init_Module-function (via ssl_util_thread_setup, which creates some thread mutexes and calls the both functions) without testing whether they have already being called or not. My question is: How does this interfere with my module? How can I ensure that only one of us (mod_ssl or my module) is calling these both functions? I cannot believe that there is no problem when my module creates some thread mutexes and mod_ssl does it too... Regards, Frank P.S.: I still think there is need for a test routine like 'ssl_is_thread_safe_maker_on()'. P.P.S.: To Nick Kew: I don't agree with Rüdigers comment. I think OpenSSL is needed in the 1st edition. :-)
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Nick Kew wrote: Unless OpenSSL nomenclature is rather confusing here, an SSL_CTX sounds like the kind of thing you would instantiate per-connection or per-request. Does your module act on a request or a connection? Maybe a bit of background reading and examination of reference implementations would be a better help for you right now. SSL_CTX_new(3): SSL_CTX_new - create a new SSL_CTX object as framework for TLS/SSL enabled functions SSL_new(3): SSL_new - create a new SSL structure for a connection The SSL_CTX is a template/configuration holder to stamp out your connection instances from. This saves configuring certificates, cipher specs, etc... for every connection. Darryl
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
> -Ursprüngliche Nachricht- > Von: Nick Kew > Gesendet: Mittwoch, 6. Dezember 2006 15:06 > An: dev@httpd.apache.org > Betreff: Re: Creating a thread safe module and the problem of > calling of 'CRYPTO_set_locking_callback' twice! > > OpenSSL is just one of thousands of libraries a module developer might > want to use, and isn't one I've drawn on for examples (as you will > no doubt infer from my ignorance of it:-) Maybe Openssl is a good reason for your 2nd edition :-). Regards Rüdiger
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Wed, 06 Dec 2006 10:22:22 +0100 Frank <[EMAIL PROTECTED]> wrote: > Your book will be available in March/April 2007, but I prefer to find > a solution somewhat earlier. :-) Someone's being very cautious with dates there. I've just finished proofreading, and am under instructions that any changes at this stage can't affect pagination because it would screw up the Index. > In case the book isn't already in print: Don't you think that this > problem should be mentioned (and solved) in it? :-) Am I the first > one who wants to write a thread-safe Apache module using OpenSSLs > crypto functions? OpenSSL is just one of thousands of libraries a module developer might want to use, and isn't one I've drawn on for examples (as you will no doubt infer from my ignorance of it:-) -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Wed, 06 Dec 2006 09:42:04 +0100 Frank <[EMAIL PROTECTED]> wrote: > William A. Rowe, Jr. wrote: > > Nick Kew wrote: > > [...] > >>Is OpenSSL not thread-safe? > > > > > > An SSL_CTX can't be cross-threaded. If the scope of use of that > > CTX is restricted to one thread at a time, then yes, OpenSSL has > > been threadsafe for a very very long time. > > You mean if I were able to create one SSL_CTX for every thread then I > do not have to use the both thread-safe-maker callbacks? Unless OpenSSL nomenclature is rather confusing here, an SSL_CTX sounds like the kind of thing you would instantiate per-connection or per-request. Does your module act on a request or a connection? -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Frank wrote: William A. Rowe, Jr. wrote: Nick Kew wrote: [...] An SSL_CTX can't be cross-threaded. If the scope of use of that CTX is restricted to one thread at a time, then yes, OpenSSL has been threadsafe for a very very long time. You mean if I were able to create one SSL_CTX for every thread then I do not have to use the both thread-safe-maker callbacks? I dont think this is true. But correct my understanding too if I am wrong. Cross-threaded might confuse someone into thinking there maybe some "apartment threading rules" to obey, there isn't. "An SSL *" can't have a method invoked on the same instance at the same time. So long as you serialize your method calls (SSL_() family) to that same instance; any thread can call that method. It is unusual to need to do so. But "SSL_CTX *" is the template context specifically designed to be shared and used across multiple-threads if needs be, providing you make correct use of the 'CRYPTO_set_locking_callback' and 'CRYPTO_set_id_callback' and friends as part of your application initialization. This allows for (amongst other things) the obviously parallel usage of SSL_new(SSL_CTX *) when creating new connections. Maybe the openssl-users list would be a better place for assistance. Darryl
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Frank wrote: Nick Kew wrote: [...] My chapter 4 discusses thread-safety and external libraries. But I know nothing about OpenSSL, and I suspect you might be seeing a problem where none exists. When the library opens I will take a look at it, but I suspect that your book will not help me in this case... [...] Your book will be available in March/April 2007, but I prefer to find a solution somewhat earlier. :-) In case the book isn't already in print: Don't you think that this problem should be mentioned (and solved) in it? :-) Am I the first one who wants to write a thread-safe Apache module using OpenSSLs crypto functions? Best regards, Frank
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
William A. Rowe, Jr. wrote: Nick Kew wrote: [...] Is OpenSSL not thread-safe? An SSL_CTX can't be cross-threaded. If the scope of use of that CTX is restricted to one thread at a time, then yes, OpenSSL has been threadsafe for a very very long time. You mean if I were able to create one SSL_CTX for every thread then I do not have to use the both thread-safe-maker callbacks? Hm, maybe this is an option. (But I must admit that I first have to find out if and how that is possible in Apache.) But anyway: As a matter of principle it would be good to know the correct solution to my original problem! Normally I don't like to bypass problems! I like to solve them! :-) Frank
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Nick Kew wrote: On Tue, 05 Dec 2006 21:45:48 +0100 Frank <[EMAIL PROTECTED]> wrote: Hi, I am developing a module (using OpenSSL) for apache 2.2.3 and wonder how to make it thread safe. Is OpenSSL not thread-safe? Yes and no. http://www.openssl.org/docs/crypto/threads.html says: "OpenSSL can safely be used in multi-threaded applications provided that at least two callback functions are set." ... and that are 'CRYPTO_set_locking_callback' and 'CRYPTO_set_id_callback'. (And these are 'global' callbacks! These are not callbacks I give as a parameter to an individual OpenSSL-function to make it called only inside the local scope of that particular function!) If it's not, how does mod_ssl deal with it? And if it is, where's the problem? I do not know what might happen if my module is calling the above two functions to set the callbacks while mod_ssl does so, too. If I tried it out everything seems to work, but this may be, because that callbacks are never get called or it happens in an environment where it makes no difference or it just works coz' I am a lucky guy or whatever So my question is: - Should I call 'ssl_util_thread_setup' too and a (maybe) second call to that function doesn't create problems? If that's a library initialisation function, call it at server startup. But it calls things like 'apr_thread_mutex_create' and the two callbacks mentioned above... It's not a real library initialisation function! Especially because it didn't test for being already called... You see I am an extremely cautious guy. (Or call it coward.) - Should there be a 'Someone_already_called_me_so_go_home'-Variable inside the function 'ssl_util_thread_setup' to prevent it from being called twice? What happens if it's called twice? What does mod_ssl do? Everthing seems to work, but maybe this is just luck. (See above.) - Do I understand something wrong? - What is the correct approach to solve this dilemma? My chapter 4 discusses thread-safety and external libraries. But I know nothing about OpenSSL, and I suspect you might be seeing a problem where none exists. When the library opens I will take a look at it, but I suspect that your book will not help me in this case... I am still confused about that topic... Frank
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Nick Kew wrote: > On Tue, 05 Dec 2006 21:45:48 +0100 > Frank <[EMAIL PROTECTED]> wrote: > >> Hi, >> I am developing a module (using OpenSSL) for apache 2.2.3 and wonder >> how to make it thread safe. > > Is OpenSSL not thread-safe? An SSL_CTX can't be cross-threaded. If the scope of use of that CTX is restricted to one thread at a time, then yes, OpenSSL has been threadsafe for a very very long time.
Re: Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
On Tue, 05 Dec 2006 21:45:48 +0100 Frank <[EMAIL PROTECTED]> wrote: > Hi, > I am developing a module (using OpenSSL) for apache 2.2.3 and wonder > how to make it thread safe. Is OpenSSL not thread-safe? If it's not, how does mod_ssl deal with it? And if it is, where's the problem? > So my question is: > - Should I call 'ssl_util_thread_setup' too and a (maybe) second call > to that function doesn't create problems? If that's a library initialisation function, call it at server startup. > - Should there be a 'Someone_already_called_me_so_go_home'-Variable > inside the function 'ssl_util_thread_setup' to prevent it from being > called twice? What happens if it's called twice? What does mod_ssl do? > - Do I understand something wrong? > - What is the correct approach to solve this dilemma? My chapter 4 discusses thread-safety and external libraries. But I know nothing about OpenSSL, and I suspect you might be seeing a problem where none exists. -- Nick Kew Application Development with Apache - the Apache Modules Book http://www.apachetutor.org/
Creating a thread safe module and the problem of calling of 'CRYPTO_set_locking_callback' twice!
Hi, I am developing a module (using OpenSSL) for apache 2.2.3 and wonder how to make it thread safe. That module uses EVP_EncryptUpdate (and friends) so I have to use 'CRYPTO_set_id_callback' and 'CRYPTO_set_locking_callback' to make the code thread safe. But I am not sure what will happen if the user configures mod_ssl! In threaded mode that module calls 'ssl_util_thread_setup' (in modules/ssl/ssl_util.c) which also calls 'CRYPTO_set_id_callback' and 'CRYPTO_set_locking_callback'! So my question is: - Should I call 'ssl_util_thread_setup' too and a (maybe) second call to that function doesn't create problems? - Should there be a 'Someone_already_called_me_so_go_home'-Variable inside the function 'ssl_util_thread_setup' to prevent it from being called twice? - Do I understand something wrong? - What is the correct approach to solve this dilemma? Any tips are more than welcome! Frank
