RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> > 3) You cannot link to the pthreads library and still use fork, and
> David, you absolutely cannot link with pthreads and still use fork()
> It doesn't work except in a few very simplistic scenarios.
> -paul
What you are saying just doesn't make any sense. I agree that it is
difficult to use fork properly in a process that creates multiple threads.
But this has nothing whatsoever to do with linking with the pthreads library
nor with compiling your code multi-threaded.
You can write code that has multiple internal models, say one that uses
'fork' the way unthreaded processes normally do and one that uses
multiple-threads, and compile it using your platform's options for compiling
a multi-threaded process. You can then select at run time whether to create
threads or to call 'fork'.
On no platform that claims POSIX compliance will you have any problems at
all. The problem you are complaining about simply does not exist.
Yes, it's difficult to use 'fork' in a process that actually creates
multiple threads. But whether or not a process *creates* multiple threads is
not a compilation issue, it's a run time issue. So it can't possibly cause
you to need to compile two versions.
I defy you to show me any platform where 'fork' breaks just because you
specify multi-threaded compiler options or link to the threading library.
You can most certainly do:
int multi_threaded;
if(multi_threaded)
{
// some code that calls pthread_create
}
else
{
// some code that calls fork
}
And compile it multi-threaded and link it to the pthreads library and you
will have *NO* issues with 'fork'.
The issues you are talking about with 'fork' are all run-time issues. None
of them require compiling two copies of a library.
DS
__
OpenSSL Project http://www.openssl.org
Development Mailing List openssl-dev@openssl.org
Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> So you had a bug in your code. So what? No bug - read this: http://www.unix.org/version2/whatsnew/threadspaper.ps : Registration of fork handlers (pthread_atfork( )). The fork handlers are routines that are to be executed in association with calls to the fork( ) function. There are three classes of fork handlers: prepare, parent, and child. Prepare fork handlers are executed prior to fork() processing, in the context of the calling thread. Parent fork handlers are executed upon completion of fork() processing in the parent, again in the context of the calling thread. Child fork handlers are executed upon completion of fork() processing in the child, in the context of the single thread initially existing in the child process. Fork handlers are envisioned as a mechanism for dealing with the problem of orphaned mutexes that can occur when a multi-threaded process calls fork(). The problem arises when threads other than the calling thread own mutexes at the time of the call to fork( ). Since the non-calling threads are not replicated in the child process, the child process is created with mutexes locked by non-existent threads. These mutexes can therefore never be unlocked. Fork handlers are intended to resolve the problem of orphaned mutexes in the following way. Prepare fork handlers can be written to lock all mutexes. In this way, orphaned mutexes are avoided, and the resources protected by the mutexes are not left in inconsistent states. This is due to the fact that the calling thread itself, which is replicated in the child process, has locked all mutexes. Thus, both the parent and child processes have all mutexes locked upon completion of fork() processing, at which time the parent and child fork handlers execute. The parent and child fork handlers unlock mutexes locked by the prepare fork handler. Fork handlers are especially useful in enabling independently-developed libraries and application programs to protect themselves from one another. A multi-threaded library can protect itself from application programs that issue fork( ) operations, possibly without even knowing that the library is multi-threaded, by providing fork handlers. Similarly, an application program can protect itself from fork( ) operations issued by library functions > 3) You cannot link to the pthreads library and still use fork, and David, you absolutely cannot link with pthreads and still use fork() It doesn't work except in a few very simplistic scenarios. -paul
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Any argument which begins with "on Linux" or (generalized) "on [platform]" is automatically suspect, regardless of whether there is any currently-extant platform which violates the assumptions put forward. For an example of why this is a problem, remember the assumption on 32- bit platforms that sizeof(int) == sizeof(void*). This caused a lot of trouble when 64-bit procs came out. For portable code, the standards are God. No matter what optimizations you can make with different assumptions. -Kyle H Sent from my iPhone On Jan 29, 2008, at 19:16, "David Schwartz" <[EMAIL PROTECTED]> wrote: On FreeBSD/OpenBSD my program outright core dumped and I could not figure out why for days and days. So you had a bug in your code. So what? Now I have two separate builds - one built with "-D_REENTRANT -DTHREADS ... -lpthread" and one without. Only with Linux do you have the freedom of which you speak. Well then that kills your point for any other platform than Linux, doesn't it? You need two builds anyway. Well lets ask this question: "does fork() replicate all Thread Local Storage data or not?" It certainly replicates the heap. It replicates all static data. It replicates all global data. But TLS? What does POSIX say about that? It's a tricky question. You will see the POSIX lords actually have something to say about it :-) I don't see what that has to do with anything. My point is simply to refute this argument: Further, on some systems you can't link with libpthread if you intend to use fork(). I have two builds of my software, one that does fork()ing and one that does pthread_create()ing. So I am trying to avoid having to have two installations of OpenSSL on every build platform. And this argument would only apply to a platform where you wouldn't otherwise two need builds, on which OpenSSL was supported, and which had POSIX threads. If there's no such platform, then this argument is bogus. 1) OpenSSL is supported, Otherwise, you won't have any OpenSSL builds. 2) Multiple threads are supported through a pthreads-like interface and are created with a pthread_create function. Otherwise, there is no issue with 'fork' and 'pthread_create' compatability. 3) You cannot link to the pthreads library and still use fork, and Otherwise, there is no compatability issue. 4) It is possible to compile a library such that it can be used by either a threaded or a non-threaded process. Otherwise, you need two builds anyway. So is there such a platform or not? If not, this argument is bogus. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> On FreeBSD/OpenBSD my program outright core dumped and I could not > figure out why for days and days. So you had a bug in your code. So what? > Now I have two separate builds - one built > with "-D_REENTRANT -DTHREADS ... -lpthread" and one without. > Only with Linux do you have the freedom of which you speak. Well then that kills your point for any other platform than Linux, doesn't it? You need two builds anyway. > Well lets ask this question: "does fork() replicate all Thread > Local Storage data or not?" It certainly replicates the heap. > It replicates all static data. It > replicates all global data. But TLS? What does POSIX say about that? > It's a tricky question. You will see the POSIX lords actually have > something > to say about it :-) I don't see what that has to do with anything. My point is simply to refute this argument: >>> Further, on some systems you can't link with libpthread >>> if you intend to use fork(). I have two builds of my >>> software, one that does fork()ing and one that does >>> pthread_create()ing. So I am trying to avoid having to have two >>> installations of OpenSSL on every build platform. And this argument would only apply to a platform where you wouldn't otherwise two need builds, on which OpenSSL was supported, and which had POSIX threads. If there's no such platform, then this argument is bogus. >1) OpenSSL is supported, Otherwise, you won't have any OpenSSL builds. >2) Multiple threads are supported through a pthreads-like interface and >are created with a pthread_create function. Otherwise, there is no issue with 'fork' and 'pthread_create' compatability. >3) You cannot link to the pthreads library and still use fork, and Otherwise, there is no compatability issue. >4) It is possible to compile a library such that it can be used by either a >threaded or a non-threaded process. Otherwise, you need two builds anyway. So is there such a platform or not? If not, this argument is bogus. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
On Tue, Jan 29, 2008 at 10:22:16PM +0200, Paul Sheer wrote: > > > > > > I find it hard to believe that there exists a platform where: > > > > On FreeBSD/OpenBSD my program outright core dumped and I could not > figure out why for days and days. Now I have two separate builds - one built > with "-D_REENTRANT -DTHREADS ... -lpthread" and one without. > Only with Linux do you have the freedom of which you speak. > > Well lets ask this question: "does fork() replicate all Thread Local Storage > data or not?" It certainly replicates the heap. It replicates all static > data. It > replicates all global data. But TLS? What does POSIX say about that? * A process shall be created with a single thread. If a multi-threaded process calls fork(), the new process shall contain a replica of the calling thread and its entire address space, possibly including the states of mutexes and other resources. Consequently, to avoid errors, the child process may only execute async-signal-safe operations until such time as one of the exec functions is called. ^[THR] [Option Start] Fork handlers may be established by means of the pthread_atfork() function in order to maintain application invariants across fork() calls. [Option End] When the application calls fork() from a signal handler and any of the fork handlers registered by pthread_atfork() calls a function that is not asynch-signal-safe, the behavior is undefined. [...] When a programmer is writing a multi-threaded program, the first described use of fork(), creating new threads in the same program, is provided by the pthread_create() function. The fork() function is thus used only to run new programs, and the effects of calling functions that require certain resources between the call to fork() and the call to an exec function are undefined. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> > > I find it hard to believe that there exists a platform where: > On FreeBSD/OpenBSD my program outright core dumped and I could not figure out why for days and days. Now I have two separate builds - one built with "-D_REENTRANT -DTHREADS ... -lpthread" and one without. Only with Linux do you have the freedom of which you speak. Well lets ask this question: "does fork() replicate all Thread Local Storage data or not?" It certainly replicates the heap. It replicates all static data. It replicates all global data. But TLS? What does POSIX say about that? It's a tricky question. You will see the POSIX lords actually have something to say about it :-) -paul
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> Further, on some systems you can't link with libpthread > if you intend to use fork(). I have two builds of my > software, one that does fork()ing and one that does > pthread_create()ing. So I am trying to avoid having to have two > installations of OpenSSL on every build platform. I find it hard to believe that there exists a platform where: 1) OpenSSL is supported, 2) Multiple threads are supported through a pthreads-like interface and are created with a pthread_create function. 3) You cannot link to the pthreads library and still use fork, and 4) It is possible to compile a library such that it can be used by either a threaded or a non-threaded process. In any event, even if there did exist a platform this bizarre, you'd just need to have two installations of OpenSSL on that one platform. There are very few, if any, platforms left that support a pthreads-like interface but have such glaring holes in their POSIX-compliance. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Paul Sheer, 2008-01-29: > "it's own data"?? - well this is exactly why I asked on this > list :-) I wanted to get a better I idea about what "it's own > data" actually means. I am growing toward a complete list > of "it's own data" that does not appear to have any chance > of races. > > The fact that yee-all aren't sure what "it's own data" precisely > means makes me a bit worried!!! :-) Why worry about this? The fact that I'm not sure (sic) about my own claim won't change *anything* about your lousy implementation. It's still relying on undocumented behavior. I think I've already made my point. I've been in a similar position before, studied a similar problem, and proposed the fix that went committed, not including at least an additional bug introduced by a MySQL employee. If you want to hear advice from someone who's been in your shoes, then please do spend some time and figure out whether your high energy approach would make any difference to your app. > > IMHO, your approach is clearly wrong: your app's fate > > is relying on undocumented behavior. It could "work" with > > a few OpenSSL library versions; but internal, sentitive > > behavior could change in future versions. Hence, I don't > > consider this a good engineering practice. > > My approach is certainly wrong by definition. > > It practice, for the OpenSSL source tree I am using, > it is 100% working. Yes, you are right - future versions > could break everything. Good to hear this. Leandro __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Leandro Santi, 2008-01-29: > I won't argue with you about using the library in an > undocumented manner; but I *do* think it'd be interesting > to get some real quantitative data: we could use it as a > basis to discuss possible future library modifications, > more compatible with your requests. One more thing, I think library usage questions don't belong to this list, so perhaps you should redirect your questions somewhere else. Library developement questions, however, are on-topic. Leandro __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> This behavior, by itself, does not necessary guarantee > that your OpenSSL library code won't race against itself, > won't corrupt its own data, or crash (hint: learn about > the MySQL case, search the archives). "it's own data"?? - well this is exactly why I asked on this list :-) I wanted to get a better I idea about what "it's own data" actually means. I am growing toward a complete list of "it's own data" that does not appear to have any chance of races. The fact that yee-all aren't sure what "it's own data" precisely means makes me a bit worried!!! :-) > IMHO, your approach is clearly wrong: your app's fate > is relying on undocumented behavior. It could "work" with > a few OpenSSL library versions; but internal, sentitive > behavior could change in future versions. Hence, I don't > consider this a good engineering practice. My approach is certainly wrong by definition. It practice, for the OpenSSL source tree I am using, it is 100% working. Yes, you are right - future versions could break everything. > I won't argue with you about using the library in an > undocumented manner; but I *do* think it'd be interesting > to get some real quantitative data: we could use it as a > basis to discuss possible future library modifications, > more compatible with your requests. I want to implement a static analyzer to work out all of OpenSSL's global variables, and incapsulate them in a global instance of the library. I have done this before with a library thus enabling me to instantiate multiple instances of a libary. I believe global variables in a library are bad practice. -paul
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Paul Sheer, 2008-01-29: > Let's say you have 1600 clients. Let's say that you have 40 threads, and > each thread > handles 40 connections. Now let's say that each thread initializes it's own > SSL_CTX structure. > > The SSL_CTX structure contains most of the data required for SSL > functionality. > Because each SSL_CTX structure has one-and-only-one thread accessing it, > there can be no contention within the SSL_CTX structure's data. > > ... This behavior, by itself, does not necessary guarantee that your OpenSSL library code won't race against itself, won't corrupt its own data, or crash (hint: learn about the MySQL case, search the archives). IMHO, your approach is clearly wrong: your app's fate is relying on undocumented behavior. It could "work" with a few OpenSSL library versions; but internal, sentitive behavior could change in future versions. Hence, I don't consider this a good engineering practice. I won't argue with you about using the library in an undocumented manner; but I *do* think it'd be interesting to get some real quantitative data: we could use it as a basis to discuss possible future library modifications, more compatible with your requests. Leandro __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Let's say you have 1600 clients. Let's say that you have 40 threads, and each thread handles 40 connections. Now let's say that each thread initializes it's own SSL_CTX structure. The SSL_CTX structure contains most of the data required for SSL functionality. Because each SSL_CTX structure has one-and-only-one thread accessing it, there can be no contention within the SSL_CTX structure's data. The next question is "is there any OTHER data (i.e. global data) within the OpenSSL library that the threads may be accessing concurrently?". The answer is "YES there are a few of these." Now of this global data, there is that which is initialized once on startup, and thereafter is "read-only", and there is that which is constantly updated. The former data is not of concern. My investigation reveals that the instances of global shared data that are modified after startup are very few. These are handled in the list in my previous emails. Of what VALUE this is, is questionable, because, it has been pointed out that locking with such an application would have minimal overhead. Myself I LIKE the idea of my code crashing so I do EVERYTHING I can to encourage my programs to crash. Each program crash is an indication of my not understanding something about my program!! :-) Saying there "MAY be something dangerous about what I am doing" is a reason TO do it because, within that "MAY" is something I don't understand about the OpenSSL library. And I don't like working with libraries that I don't understand! :^) Further, on some systems you can't link with libpthread if you intend to use fork(). I have two builds of my software, one that does fork()ing and one that does pthread_create()ing. So I am trying to avoid having to have two installations of OpenSSL on every build platform. -paul On Jan 25, 2008 4:45 PM, Leandro Santi <[EMAIL PROTECTED]> wrote: > Tomas Mraz, 2008-01-24: > > So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given > > that there won't be any static and/or global variables in the OpenSSL > > code called is 100% safe thing if the threads do not share any data > > manipulated within the OpenSSL library. > > As mentioned in the docs, multithreaded OpenSSL needs special > application support, period. Not providing this means you'll > get undefined/undesirable results. > > Old MySQL versios did try this approach (i.e. using the library > in an undocumented way). Perhaps it sort-of worked for them > while they developed the SSL support for the database engine, > but newer MySQL/OpenSSL combinations didn't work at all (for > example, MySQL-4.0.23a+OpenSSL-0.9.7c). They fixed it, albeit a > few years down the road. > > Separately, I'd suggest a different development approach: first, > implement OpenSSL locking support. Do some measurements, with > and without locking. I'd be interested to get some quantitative > evidence before proceeding with this thread. > > Leandro > __ > OpenSSL Project http://www.openssl.org > Development Mailing List openssl-dev@openssl.org > Automated List Manager [EMAIL PROTECTED] >
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
On Mon, Jan 28, 2008 at 02:22:09PM -0800, David Schwartz wrote: > > > errno is stored in Thread Local Storage (TLS). You can't link to the > > global errno anymore. > > For a single-threaded process, there is no distinction between thread-local > storage and a global variable. For a multi-threaded process, there is. There is no "global" variable named errno, it only exist in the TLS. You could say that because there is only 1 TLS, that it's global, and it acts that way. But it's not really the same as a normal global variable. You can't access the variables in the same manner you access other global variables. > The same code can have a different semantic meaning depending upon whether > the process is single-threaded or multi-threaded. > > > Using "extern int errno" will actually result in a linking error: > > /usr/bin/ld: errno: TLS definition in /lib/libc.so.6 section > > .tbss mismatches non-TLS reference > > My point was simply that there is a difference between single-threaded and > multi-threaded compilation. In one case, 'errno' refers to a variable that > is process global (since there is only one thread). In the other case, it > does not. In case of errno on a glibc system with NPTL there is no difference in the compiled code. You always get the same function call. Let's assume for a moment that openssl is not "compiled for multi thread", whatever you mean with that exactly. Even when the application is using threads, when openssl tries to use errno it will get the one for the current thread. > That these two different semantics are implemented by the same code doesn't > change the fact that the semantics are different. In fact, it makes the > important point that the semantics of code can be changed by whether the > process is single-threaded or multi-threaded. I'm not at all saying that the semantics are the same. All I'm saying is that in case of errno on a recent glibc there is no difference in the compiled code depending on wether your application or library is multi-threaded or not. > On many Linux distributions, it is perfectly acceptable to do the following: > > #ifdef _REENTRANT > // inefficient thread-safe code > #else > // more efficient code that assumes 'errno' is process-global > #endif On many Linux distributions it's required that if such code is in a library that it's compiled using -D_REENTRANT for the packages they ship. With other words, if the library can be compiled reentrant, that's what we want. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> errno is stored in Thread Local Storage (TLS). You can't link to the > global errno anymore. For a single-threaded process, there is no distinction between thread-local storage and a global variable. For a multi-threaded process, there is. The same code can have a different semantic meaning depending upon whether the process is single-threaded or multi-threaded. > Using "extern int errno" will actually result in a linking error: > /usr/bin/ld: errno: TLS definition in /lib/libc.so.6 section > .tbss mismatches non-TLS reference My point was simply that there is a difference between single-threaded and multi-threaded compilation. In one case, 'errno' refers to a variable that is process global (since there is only one thread). In the other case, it does not. That these two different semantics are implemented by the same code doesn't change the fact that the semantics are different. In fact, it makes the important point that the semantics of code can be changed by whether the process is single-threaded or multi-threaded. On many Linux distributions, it is perfectly acceptable to do the following: #ifdef _REENTRANT // inefficient thread-safe code #else // more efficient code that assumes 'errno' is process-global #endif This is because many distributions only guarantee POSIX compliance if you specify '-pthread' and '-pthread' always defines _REENTRANT. You have to follow the rules if you want the platform to follow the rules. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
On Mon, Jan 21, 2008 at 05:34:43PM -0800, David Schwartz wrote: > > - there is no difference between > > multithreaded and non-multithreaded _compilation_ (surely not for errno > > and malloc). > > Really? So 'errno' refers to a process global in both cases?! (Note that I > said the "definition", not the implementation.) If your glibc (only) supports NPTL, which is the case for recent versions, errno is defined like this: # define errno (*__errno_location ()) errno is stored in Thread Local Storage (TLS). You can't link to the global errno anymore. Using "extern int errno" will actually result in a linking error: /usr/bin/ld: errno: TLS definition in /lib/libc.so.6 section .tbss mismatches non-TLS reference Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Tomas Mraz, 2008-01-24: > So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given > that there won't be any static and/or global variables in the OpenSSL > code called is 100% safe thing if the threads do not share any data > manipulated within the OpenSSL library. As mentioned in the docs, multithreaded OpenSSL needs special application support, period. Not providing this means you'll get undefined/undesirable results. Old MySQL versios did try this approach (i.e. using the library in an undocumented way). Perhaps it sort-of worked for them while they developed the SSL support for the database engine, but newer MySQL/OpenSSL combinations didn't work at all (for example, MySQL-4.0.23a+OpenSSL-0.9.7c). They fixed it, albeit a few years down the road. Separately, I'd suggest a different development approach: first, implement OpenSSL locking support. Do some measurements, with and without locking. I'd be interested to get some quantitative evidence before proceeding with this thread. Leandro __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
Paul Sheer wrote: Locking with no contention is not "pretty expensive", it's darn near free. Oh? If this is true it changes things somewhat. But I must say that I believe that no-one has ever used OpenSSL with 10'000 concurrent SSL objects. So I'm not going to take the chance that there will be any performance degradation. I'd rather have the GUARANTEE that there is no lock contention AT ALL. In my opinion what you are doing is going to result in some subtle bug that is going to be impossible to track down if there is even a single location where two threads can step on each other's toes. Locks are placed into the code when the developer of the code believes it is not safe to perform an operation in a multi-threaded environment without the guarantee that no other thread can be accessing or manipulating the same data. If your application is designed correctly and there is in fact no contention between the threads, then the cost of obtaining and releasing locks will be inconsequential. You can do what you will but as someone who has years of experience tracking down bugs in live systems caused by races between threads that were not prevented by locks, I think you are being foolish. It is not worth the cost of a production system going down or valuable data being lost or corrupted. Jeffrey Altman Secure Endpoints Inc. smime.p7s Description: S/MIME Cryptographic Signature
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
David Jacobson wrote: > I'm only familiar with Solaris. In that system the real stuff > in a mutex is a byte about 12 bytes into the lock structure. > On SPARC the mutex_lock function accesses it with an LDSTUB > instruction, which is a special atomic instruction that loads > the old value into a register, and stores 0xff into it. That's pretty typical of what happens on most platforms. An uncontended lock/release will typically result in only a single atomic operation and the possible cache miss of the actually lock guts. Last I measured on x86 with modern hardware (this was P4/Netburst), the cost of the atomic operation is the same whether or not it results in a cache miss. This was probably due to the pipelines being so deep. I would expect both numbers to be much less on a more modern Core2, but perhaps the gap between them has grown. (Fewer cycles performing the atomic operation means less time to hide the cache miss.) Most of the cost is believed to be due to pipeline flush. > If that byte is in the same cache line as some other variable > that gets written, and if another processor writes that other stuff, > the cache line will be owned by that other processor, For just this reason, it is common to put the lock guts byte on its own cache line or to get the equivalent affect another way. On modern x86 systems, the cost of the atomic operation generally swamps the cost of the cache miss, which is mitigated anyway by prefetching. > and an > uncontended mutex_lock can be terribly expensive. To some extent, it depends on what you're comparing it to. I think it's unreasonable to describe a single atomic operation, even if a cache miss is guaranteed, as "terribly expensive". You'd have to work pretty hard to make that matter, and certainly it shrinks to nothing in comparison to anything OpenSSL might do while it holds the lock. In the vast majority of ordinary cases, the cost of an uncontended lock is effectively zero. You have to spin in tight loops to measure it. In any event, in the OP's case, he won't have the CPU ping-pong issue anyway. He never has a case where two threads manipulate the same object, so they won't be ping-ponging the same lock either. It doesn't seem possible for him to get the kind of tight loop that would make the cost measurable. It would make sense for him to measure in his application though. There's always the off chance that for some odd reason that isn't easily forseeable this is expensive in his particular situation. Engineering intuition should guide the original design, but you should always measure to make sure your intuition was right. Plus, that's needed to develop intuition in the first place. My point is that the OP is proceeding on a path that conflicts with sensible engineering. He would need a very unusual problem or some very unusual measurements of a more obvious solution to justify the risk his path entails. Or, of course, if this was a research or experimental approach rather than a real-world application. In that case, it's perfectly sensible to intentionally try things that are atypical to see what happens. However, he should have a baseline to show that the overhead he's trying to minimize is at least measurable first. Otherwise, he can never show a reduction. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
On Thu, 2008-01-24 at 09:11 -0800, David Schwartz wrote: > > > Really? So 'errno' refers to a process global in both cases?! > > > (Note that I > > > said the "definition", not the implementation.) > > > Maybe we didn't understand each other - I don't say, that glibc without > > multithread support and with it is the same. I say that linking single > > threaded library which is simply reentrant but doesn't use any locking > > itself to a multithreaded app is 100% safe thing to do. > > I think you are confusing two different definitions of "single threaded". > You may mean by "single threaded" that no threading functions are ever > called. This is a source code issue. The source either calls threading > functions or it doesn't. > > But what the OP and I mean by single threaded is *compiled* single threaded. > That is, without specifying whatever options the compiler wants to make code > multi threaded. I know that other operating systems/compilers/C runtime libraries are different. And in fact I don't know in what environment Paul Sheer's application will be running, but as I wrote above on current gcc+glibc as used by most widespread Linux distributions there are no extra options which are needed if you want to make code multithreaded. The glibc's are always compiled to be threadsafe (with locking stubs in place) which are replaced with real locking code when libpthread is linked in. So gcc threadedapp.c -o threadedapp -lpthread is everything what is needed to compile and link multithreaded executable. Again, I know that the situation with other compilers/libcs/OSes can be and is very different. -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
I'm only familiar with Solaris. In that system the real stuff in a mutex is a byte about 12 bytes into the lock structure. On SPARC the mutex_lock function accesses it with an LDSTUB instruction, which is a special atomic instruction that loads the old value into a register, and stores 0xff into it. If that byte is in the same cache line as some other variable that gets written, and if another processor writes that other stuff, the cache line will be owned by that other processor, and an uncontended mutex_lock can be terribly expensive. -- David Jacobson David Schwartz <[EMAIL PROTECTED]> wrote: > > Locking with no contention is not "pretty expensive", it's darn near > > free. > On systems with only one processor and nothing like hyperthreading. Did you miss the "with no contention" part? An uncontended lock costs about the same on an SMP system as on an MP system. AFAIK, hyperthreading doesn't affect the cost of uncontended locks. An uncontended lock typically results in one atomic operation (which doesn't actually locked any busses anymore) and possibly one additional cache miss (assuming the lock was recently last held by another CPU). Other costs are totally drowned out by these two. As for a contended lock, it's probably also typically less expensive on a multi-CPU system (though contention is more likely, so it's kind of a bogus comparison). Contention will typically result in more context switches on a single CPU system, and the cost of context switches is enormous compared to the other costs we're measuring here. Hyperthreading likely reduces the cost of a contended lock because the other virtual execution unit gains the use of the execution units the contending thread is not using and contention across virtual execution units in the same core is typically less expensive than across the FSB. I can't think of any obvious reason hyperthreading would have any significant affect on the cost of contention, unless you're talking about broken spinlocks that don't properly relax the CPU (stealing execution resources from the virtual core that's doing useful work). Obviously, broken spinlocks will cause problems on an HT machine, but they should all be fixed by now. In any event, he's talking about a situation where he does everything he can to reduce contention to zero. So the only issues left are what the cost of uncontended locks is (to decide whether it's worth eliminating the locks entirely) and what the consequences are if he misses a case (disaster if he removes the lock, nothing if he doesn't). DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> > On systems with only one processor and nothing like hyperthreading. > > Did you miss the "with no contention" part? No. I didn't realize you meant it as 'no possible contention.' /r$ -- STSM, DataPower Chief Programmer WebSphere DataPower SOA Appliances http://www.ibm.com/software/integration/datapower/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
I'm replying to David's email off-list... -paul On Jan 24, 2008 8:44 PM, David Schwartz <[EMAIL PROTECTED]> wrote: > > > But I must say that I believe that no-one has ever used OpenSSL > > with 10'000 concurrent SSL objects. > > Umm, what?! We've last tested to almost 16,384. Our first test to 10,000 > was > many years ago, on servers with Pentium processors and 128MB of RAM. > We've >
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> > Locking with no contention is not "pretty expensive", it's darn near > > free. > On systems with only one processor and nothing like hyperthreading. Did you miss the "with no contention" part? An uncontended lock costs about the same on an SMP system as on an MP system. AFAIK, hyperthreading doesn't affect the cost of uncontended locks. An uncontended lock typically results in one atomic operation (which doesn't actually locked any busses anymore) and possibly one additional cache miss (assuming the lock was recently last held by another CPU). Other costs are totally drowned out by these two. As for a contended lock, it's probably also typically less expensive on a multi-CPU system (though contention is more likely, so it's kind of a bogus comparison). Contention will typically result in more context switches on a single CPU system, and the cost of context switches is enormous compared to the other costs we're measuring here. Hyperthreading likely reduces the cost of a contended lock because the other virtual execution unit gains the use of the execution units the contending thread is not using and contention across virtual execution units in the same core is typically less expensive than across the FSB. I can't think of any obvious reason hyperthreading would have any significant affect on the cost of contention, unless you're talking about broken spinlocks that don't properly relax the CPU (stealing execution resources from the virtual core that's doing useful work). Obviously, broken spinlocks will cause problems on an HT machine, but they should all be fixed by now. In any event, he's talking about a situation where he does everything he can to reduce contention to zero. So the only issues left are what the cost of uncontended locks is (to decide whether it's worth eliminating the locks entirely) and what the consequences are if he misses a case (disaster if he removes the lock, nothing if he doesn't). DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> But I must say that I believe that no-one has ever used OpenSSL > with 10'000 concurrent SSL objects. Umm, what?! We've last tested to almost 16,384. Our first test to 10,000 was many years ago, on servers with Pentium processors and 128MB of RAM. We've tested on operating systems from Windows NT server to Linux. This is one of the required tests before we certify any platform for use with our software. We've been offering web server and chat server software rated for 10,000 concurrent secure connections (using OpenSSL) since the year 2000. > So I'm not going to take the chance > that there will be any performance degradation. > I'd rather have the GUARANTEE that there is no lock > contention AT ALL. This makes no sense. Suppose you do everything you do but compile OpenSSL normally, with locks. If there is any lock contention, you might hypothetically get some performance degradation. If you compile OpenSSL without locks, and you encounter a situation that would normally cause lock contention, you *blow* *up*. So how does converting contention from a performance loss to a disaster help? Removing lock *CONTENTION* is not the same thing as removing locks. In any event, what you are doing makes no sense for at least one more reason. When I explained that locks without contention were darned near free, you seemed surprised. This means you never measured the cost of these locks. But then when I explained that this assumption was likely an example of premature optimization, you agreed with me. What you are doing is a painful and complex optimization with several significant risks if it's not done correctly. Since you have no demonstrated need for this optimization, why are you doing it? Assuming this is a real-world application rather than a theoretical exercise, it makes no sense. I think you are making two fundamental mistakes. First, you are assuming the cost of lock contention is high without any evidence. Second, you are confusing the cost of an uncontended lock with the cost of contention. Try it the way everyone else is doing it first. They may know something. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> Locking with no contention is not "pretty expensive", it's darn near free. On systems with only one processor and nothing like hyperthreading. /r$ -- STSM, DataPower Chief Programmer WebSphere DataPower SOA Appliances http://www.ibm.com/software/integration/datapower/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> > > So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given > > that there won't be any static and/or global variables in the OpenSSL > > code called is 100% safe thing if the threads do not share any data > > manipulated within the OpenSSL library. > > But that's not what he's doing. He's compiling OpenSSL single-threaded, > that > is, without specifying the compiler and configuration options required for > multi-threaded code. That this disables locking function calls is just one > thing it does. > The other thing it does is fail to define errno as (*__errno()). Because OpenSSL is careful not to use many of the more threadishly two-faced standard library calls, these are actually the ONLY two things it disables. On all systems, the multithreaded compiler options do nothing but, 1. link certain library calls with Thread Local Storage versions, 2. #define certain other library calls, and 3. handle main() initialization and fork() and exit() differently. 4. Use a thread-safe malloc. OpenSSL never used any of the calls in point 1. or 2. and is miles away from 3. As for 4., this is dealt with in my previous email. (Well - I sure HOPE everything I'm saying here is true!! :-) Locking with no contention is not "pretty expensive", it's darn near free. Oh? If this is true it changes things somewhat. But I must say that I believe that no-one has ever used OpenSSL with 10'000 concurrent SSL objects. So I'm not going to take the chance that there will be any performance degradation. I'd rather have the GUARANTEE that there is no lock contention AT ALL. > This assumption is a common cause of premature optimization and buggy and > hard to maintain code. :-) You have my full agreement here! I do agree that it's at least possibly interesting to provide alternate code > paths that minimize locking for possible environments where there's some > value to minimizing locking. However, I bet you would have to work awfully > hard to create a real world case where the locking was expensive that > wasn't > easily fixed by the same application-level changes you would have to do > anyway to make this work. I intend to work that hard!! Kind regards -paul
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL-possible bug???)
> > Really? So 'errno' refers to a process global in both cases?! > > (Note that I > > said the "definition", not the implementation.) > Maybe we didn't understand each other - I don't say, that glibc without > multithread support and with it is the same. I say that linking single > threaded library which is simply reentrant but doesn't use any locking > itself to a multithreaded app is 100% safe thing to do. I think you are confusing two different definitions of "single threaded". You may mean by "single threaded" that no threading functions are ever called. This is a source code issue. The source either calls threading functions or it doesn't. But what the OP and I mean by single threaded is *compiled* single threaded. That is, without specifying whatever options the compiler wants to make code multi threaded. > Of course the > glibc itself is a different piece of code - the code of things like > malloc and errno will be definitely different when glibc is compiled > without any support for threads and locking. And I am all the time > talking about dynamic linking to glibc and not static. The problem is that when you dynamically link to glibc, you cannot control what version of glibc you get. If a newer version of glibc is available, you will get the newer version. You can reasonably rely on a newer version maintaining the same guarantees as the version you tested with. If it didn't, it wouldn't be declared compatible. However, you cannot reasonably rely on assumptions you made still being true. There is no inherent reason code not compiled multi-threaded and code compiled multi-threaded have to be compatible in *any* way. There is no reason code compiled re-entrant and code compiled multi-threaded have to be compatible in any way either. To the extent they do, it's simply a happy accident. > So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given > that there won't be any static and/or global variables in the OpenSSL > code called is 100% safe thing if the threads do not share any data > manipulated within the OpenSSL library. But that's not what he's doing. He's compiling OpenSSL single-threaded, that is, without specifying the compiler and configuration options required for multi-threaded code. That this disables locking function calls is just one thing it does. > As locking can be pretty expensive in terms of CPU cycles Locking with no contention is not "pretty expensive", it's darn near free. This assumption is a common cause of premature optimization and buggy and hard to maintain code. > it is > desirable to allow using OpenSSL this way so to remove all static and/or > global variables in its code or at least isolate them and allow using > OpenSSL code without them being touched. Perhaps, but that would still require that you compile the library using whatever multi-threaded options your platform, compiler, and its configuration require. It would then by multi-threaded but wouldn't have internal locks. I do agree that it's at least possibly interesting to provide alternate code paths that minimize locking for possible environments where there's some value to minimizing locking. However, I bet you would have to work awfully hard to create a real world case where the locking was expensive that wasn't easily fixed by the same application-level changes you would have to do anyway to make this work. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given > that there won't be any static and/or global variables in the OpenSSL > code called is 100% safe thing [>] if [<] the threads do not share any data > manipulated within the OpenSSL library. Quite a big "if" there!! :-) Well my list of mods are now as follows: 1. BIO_new() with my own BIO objects. 2. RAND_set_rand_method() with my own RAND object. 3. CRYPTO_set_ex_data_implementation() to dummy implementation. 4. Set SSL_SESS_CACHE_NO_INTERNAL | SSL_SESS_CACHE_SERVER | SSL_SESS_CACHE_NO_AUTO_CLEAR. 5. Init exactly one SSL_CTX context per thread. 6. ERR_set_implementation() my own error implemention which keeps an error state per context(thread). 7. CRYPTO_set_mem_functions() my own thread-safe functions. 8. Build OpenSSL with no-threads no-dso no-comp (no-comp is for an 80-byte memory leak in the comp stack) and of course -DOPENSSL_NO_LOCKING. 9. Add a couple of lines to the lh_hash code to assert() if lh_hash is used AT ALL after SSL_library_init() is called. Just for assurance. The result is that: A. OpenSSL does not ever look at errno, B. accesses no global vars after initialization. C. appears to be stable at 50 concurrent connections and 7 concurrent sessions/threads all running under the same process. Point B. is still not absolutely guaranteed - HELP!! ;-) -paul > As locking can be pretty expensive in terms of CPU cycles it is > desirable to allow using OpenSSL this way so to remove all static and/or > global variables in its code or at least isolate them and allow using > OpenSSL code without them being touched. Yep! -paul > -- > Tomas Mraz > No matter how far down the wrong road you've gone, turn back. > Turkish proverb > > __ > OpenSSL Project http://www.openssl.org > Development Mailing List openssl-dev@openssl.org > Automated List Manager [EMAIL PROTECTED] >
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
On Mon, 2008-01-21 at 17:34 -0800, David Schwartz wrote: > > On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote: > > > > Most definitely not. At a minimum, the definition of things > > > like 'errno' and > > > 'malloc' might be different between a multithreaded build and a > > > non-multithreaded build. There is no supported way to combine > > > multithreaded > > > code and code that was not compiled to be multithreaded. > > > > It may happen to work, but that's a lousy way to make security-sensitive > > > software. > > > Definitely not true on gcc+glibc > > Umm, definitely true. > > > - there is no difference between > > multithreaded and non-multithreaded _compilation_ (surely not for errno > > and malloc). > > Really? So 'errno' refers to a process global in both cases?! (Note that I > said the "definition", not the implementation.) Maybe we didn't understand each other - I don't say, that glibc without multithread support and with it is the same. I say that linking single threaded library which is simply reentrant but doesn't use any locking itself to a multithreaded app is 100% safe thing to do. Of course the glibc itself is a different piece of code - the code of things like malloc and errno will be definitely different when glibc is compiled without any support for threads and locking. And I am all the time talking about dynamic linking to glibc and not static. So IMO what Paul Sheer is doing - disabling all locking in OpenSSL given that there won't be any static and/or global variables in the OpenSSL code called is 100% safe thing if the threads do not share any data manipulated within the OpenSSL library. As locking can be pretty expensive in terms of CPU cycles it is desirable to allow using OpenSSL this way so to remove all static and/or global variables in its code or at least isolate them and allow using OpenSSL code without them being touched. -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> Removing all the problems you can find is simply not a reliable way to > develop software. You have to design the software such that there aren't > problems, then remove any that slipped through. You can't use testing as the > way to create the guarantee in the first place. Heh :-) I totally agree with that!! -paul On Jan 22, 2008 8:47 PM, David Schwartz <[EMAIL PROTECTED]> wrote: > > > Guaranteed to work? Who's doing the indemnification? > > The point of a guarantee is that it is much less likely to change on > another > machine or if a library is upgraded and compatability is claimed. Of > course, > things can still go wrong. > > When bugs are fixed in a library or a new version claims compatability > with > an old version, every effort it made to ensure that whatever guarantees > were > provided with the old version are still met by the new version. If you > stick > to those guarantees, and they are sufficient that you don't need > additional > assumptions, you are much less likely to run into a problem if the library > is upgraded or otherwise different from the one you tested with. > > > Security's all about trade-offs. If you can make some simplifying > > assumptions that cut out large parts of code you might well be better > off. > > The risk that a dynamic library on another machine will break his > assumptions is greater than any benefit from eliminating code. This is > especially true in this case, where it's hard to imagine any risk created > by > adding locks (other than deadlock, which would seem to be very unlikely to > pose a security risk). > > Removing all the problems you can find is simply not a reliable way to > develop software. You have to design the software such that there aren't > problems, then remove any that slipped through. You can't use testing as > the > way to create the guarantee in the first place. > > The design has to be bug-resistant in the first place. > > To use an engineering analogy, imagine if you design a bridge. You fail to > specify what type of steel to use, but when you test, you pick a very > high-quality steel. Did you validate the bridge design? > > The answer is no, because you tested with only one steel, and the design > isn't limited to that one steel. > > On the flip side, if you design to the guarantees the steel manufacturer > provides, specify steel that meets those guarantees, and test with that > steel, then you have a validated design. > > Responding to Paul: > > > Since OpenSSL can be compiled as a non-locking, > > single-threaded library, I'm trying to answer the question: > > If I have one context per thread, will these contexts trash > > each others data when OpenSSL is compiled with no locking. > > The first 4 points of my list are the trashings that could take place. > > > Now I just need the complete list :-) > > You missed step 1. As far as I know, *no* platform provides that code > compiled single-threaded is thread-safe in fundamental ways. For example, > AFAIK, there is no guarantee that 'malloc' won't be "optimized" in a > single-threaded compile in ways that break horribly on multi-threaded > code. > > You need the completel list for your *platform* and an assurance that no > new > entries will be added to that list on any platform your code will compile > on > or run on. > > Absent such a guarantee, your attempt is a disaster waiting to happen even > if it works in the first place. > > DS > > > __ > OpenSSL Project http://www.openssl.org > Development Mailing List openssl-dev@openssl.org > Automated List Manager [EMAIL PROTECTED] >
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> Guaranteed to work? Who's doing the indemnification? The point of a guarantee is that it is much less likely to change on another machine or if a library is upgraded and compatability is claimed. Of course, things can still go wrong. When bugs are fixed in a library or a new version claims compatability with an old version, every effort it made to ensure that whatever guarantees were provided with the old version are still met by the new version. If you stick to those guarantees, and they are sufficient that you don't need additional assumptions, you are much less likely to run into a problem if the library is upgraded or otherwise different from the one you tested with. > Security's all about trade-offs. If you can make some simplifying > assumptions that cut out large parts of code you might well be better off. The risk that a dynamic library on another machine will break his assumptions is greater than any benefit from eliminating code. This is especially true in this case, where it's hard to imagine any risk created by adding locks (other than deadlock, which would seem to be very unlikely to pose a security risk). Removing all the problems you can find is simply not a reliable way to develop software. You have to design the software such that there aren't problems, then remove any that slipped through. You can't use testing as the way to create the guarantee in the first place. The design has to be bug-resistant in the first place. To use an engineering analogy, imagine if you design a bridge. You fail to specify what type of steel to use, but when you test, you pick a very high-quality steel. Did you validate the bridge design? The answer is no, because you tested with only one steel, and the design isn't limited to that one steel. On the flip side, if you design to the guarantees the steel manufacturer provides, specify steel that meets those guarantees, and test with that steel, then you have a validated design. Responding to Paul: > Since OpenSSL can be compiled as a non-locking, > single-threaded library, I'm trying to answer the question: > If I have one context per thread, will these contexts trash > each others data when OpenSSL is compiled with no locking. > The first 4 points of my list are the trashings that could take place. > Now I just need the complete list :-) You missed step 1. As far as I know, *no* platform provides that code compiled single-threaded is thread-safe in fundamental ways. For example, AFAIK, there is no guarantee that 'malloc' won't be "optimized" in a single-threaded compile in ways that break horribly on multi-threaded code. You need the completel list for your *platform* and an assurance that no new entries will be added to that list on any platform your code will compile on or run on. Absent such a guarantee, your attempt is a disaster waiting to happen even if it works in the first place. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> > How are you sure of this ? Did you manually remove the object code from > the library, or place an abort(); call at the points -DPURIFY is listed > to you can be sure that execution never gets there. > Actually, I am trying to create an application that has one context per thread, where each thread handles many SSL connections. Since OpenSSL can be compiled as a non-locking, single-threaded library, I'm trying to answer the question: If I have one context per thread, will these contexts trash each others data when OpenSSL is compiled with no locking. The first 4 points of my list are the trashings that could take place. Now I just need the complete list :-) -paul > > > 6. Build OpenSSL with no-threads and -DOPENSSL_NO_LOCKING > > Am I doing anything completely insane here? > > If you are rebuilding OpenSSL why don't you throw in -DPURIFY and piece > by piece undo each point you listed until the problem comes back. That > would be far more useful to the mailing-list than all this other > information. > > > If I read things correctly, "You have a hybrid threaded app" but OpenSSL > does not have any threading support. Maybe this is okay since all the > things you disabled might end up causing no thread sensitive code path > to execute. To "no-threads" option is intended for a single threaded > execution environment. > > Darryl > __ > OpenSSL Project http://www.openssl.org > Development Mailing List openssl-dev@openssl.org > Automated List Manager [EMAIL PROTECTED] >
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> IMO, writing security software by doing something that is specifically not > documented or guaranteed to work and then trying to fix every problem it > creates (at least, that you can find) is completely insane. Guaranteed to work? Who's doing the indemnification? Security's all about trade-offs. If you can make some simplifying assumptions that cut out large parts of code you might well be better off. /r$ -- STSM, DataPower Chief Programmer WebSphere DataPower SOA Appliances http://www.ibm.com/software/integration/datapower/ __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> > IMO, writing security software by doing something that is specifically not > documented or guaranteed to work and then trying to fix every problem it > creates (at least, that you can find) is completely insane. > Ok, I managed to "find" another problem: error setting/getting (eg. ERR_clear_error) is done on a per-thread basis - requiring proper locking to access a global hash table. -paul
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Paul Sheer wrote: Well my hybrid threaded app now seems to be stable - even under extreme loads. 2. Use my own RAND object (RAND_set_rand_method) so that OpenSSL does not try lock static globals. How are you sure of this ? Did you manually remove the object code from the library, or place an abort(); call at the points -DPURIFY is listed to you can be sure that execution never gets there. 6. Build OpenSSL with no-threads and -DOPENSSL_NO_LOCKING Am I doing anything completely insane here? If you are rebuilding OpenSSL why don't you throw in -DPURIFY and piece by piece undo each point you listed until the problem comes back. That would be far more useful to the mailing-list than all this other information. If I read things correctly, "You have a hybrid threaded app" but OpenSSL does not have any threading support. Maybe this is okay since all the things you disabled might end up causing no thread sensitive code path to execute. To "no-threads" option is intended for a single threaded execution environment. Darryl __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> Am I doing anything completely insane here? IMO, writing security software by doing something that is specifically not documented or guaranteed to work and then trying to fix every problem it creates (at least, that you can find) is completely insane. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
Well my hybrid threaded app now seems to be stable - even under extreme loads. Here is what I did for others to refer: Comments *most* welcome. These steps allow me to link both builds of my program with the same non-threaded OpenSSL build. I.e. both the fork() and pthread_create() builds of my software. 1. Use my own BIO object (BIO_new) so that OpenSSL does not use socket ops - allows me to use the library asyncronously and avoids errno+threading issues. 2. Use my own RAND object (RAND_set_rand_method) so that OpenSSL does not try lock static globals. 3. Use my own EXDATA object (CRYPTO_set_ex_data_implementation(my_impl)) - this is a problem because the st_CRYPTO_EX_DATA_IMPL object is not declared in the header - you have to copy and paste it from the OpenSSL source. My EXDATA object is a dummy opject - it does nothing and asserts when you try do a dup. I'm not using ex_data in my app - and quite honestly I'm not really sure what it's for. 4. Disable all OpenSSL caching of sessions - (SSL_CTX_set_session_cache_mode (ctx, SSL_SESS_CACHE_NO_INTERNAL | SSL_SESS_CACHE_SERVER | SSL_SESS_CACHE_NO_AUTO_CLEAR)) use the callbacks ( SSL_CTX_sess_set_new_cb etc.) and handle session caching and session expiry myself. 5. Declare only one SSL_CTX context per thread. 6. Build OpenSSL with no-threads and -DOPENSSL_NO_LOCKING Am I doing anything completely insane here? -paul
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL -possible bug???)
> On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote: > > Most definitely not. At a minimum, the definition of things > > like 'errno' and > > 'malloc' might be different between a multithreaded build and a > > non-multithreaded build. There is no supported way to combine > > multithreaded > > code and code that was not compiled to be multithreaded. > > It may happen to work, but that's a lousy way to make security-sensitive > > software. > Definitely not true on gcc+glibc Umm, definitely true. > - there is no difference between > multithreaded and non-multithreaded _compilation_ (surely not for errno > and malloc). Really? So 'errno' refers to a process global in both cases?! (Note that I said the "definition", not the implementation.) > So it is pretty safe to use non-multithreaded libraries (if > they are reentrant) in multithreaded applications on gcc+glibc. Of > course at link time and especially run time, the code being executed in > glibc is very different. The problem is that it is neither specified to work nor guarantee to work. You may enjoy writing security-sensitive software that "just happens to work" where you have to retest your assumptions with every new version of the system libraries, but I don't. I hope you don't dynamically-link such a thing. You have no way of knowing if the glibc libraries you might happen to run-time link to will work. When you use "features" that aren't guaranteed, a newer version may break the assumptions and still claim compatability. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL - possible bug???)
On Mon, Jan 21, 2008 at 09:24:34AM +0100, Tomas Mraz wrote: > On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote: > > > I should be able to create a multithreaded application using > > > a non-multithreaded openssl build provided that I have an ssl > > > context per thread. > > > > Most definitely not. At a minimum, the definition of things like 'errno' and > > 'malloc' might be different between a multithreaded build and a > > non-multithreaded build. There is no supported way to combine multithreaded > > code and code that was not compiled to be multithreaded. > > > > It may happen to work, but that's a lousy way to make security-sensitive > > software. > > Definitely not true on gcc+glibc - there is no difference between > multithreaded and non-multithreaded _compilation_ (surely not for errno > and malloc). In Debian we have a policy of compiling all libraries with -D_REENTRANT. As far as I know this was needed for LinuxThreads support. I'm not sure it's still needed since we switch to NPTL. There was a time when -D_REENTRANT had some effects, I'm not sure it still does. Kurt __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL - possible bug???)
On Sun, 2008-01-20 at 11:59 -0800, David Schwartz wrote: > > I should be able to create a multithreaded application using > > a non-multithreaded openssl build provided that I have an ssl > > context per thread. > > Most definitely not. At a minimum, the definition of things like 'errno' and > 'malloc' might be different between a multithreaded build and a > non-multithreaded build. There is no supported way to combine multithreaded > code and code that was not compiled to be multithreaded. > > It may happen to work, but that's a lousy way to make security-sensitive > software. Definitely not true on gcc+glibc - there is no difference between multithreaded and non-multithreaded _compilation_ (surely not for errno and malloc). So it is pretty safe to use non-multithreaded libraries (if they are reentrant) in multithreaded applications on gcc+glibc. Of course at link time and especially run time, the code being executed in glibc is very different. -- Tomas Mraz No matter how far down the wrong road you've gone, turn back. Turkish proverb __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: Static global - bug? (Re: Two valgrind warnings in OpenSSL - possible bug???)
> Most definitely not. ... Yes you are right. To qualify what I want to do: openssl does a lot of sharing and locking. I would like to use the minimum by have one context per thread. I use my own session cache so there 'should' not need to be anything else that needs sharing. Further, I use my own BIO buffers and rand object. So in my app, openssl does not go to the operating system for anything. Can you please give me some hints what I should look at changing in openssl to do this? Paul Sent via my BlackBerry from Vodacom - let your email find you! -Original Message- From: "David Schwartz" <[EMAIL PROTECTED]> Date: Sun, 20 Jan 2008 11:59:00 To: Subject: RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL - possible bug???) > I should be able to create a multithreaded application using > a non-multithreaded openssl build provided that I have an ssl > context per thread. Most definitely not. At a minimum, the definition of things like 'errno' and 'malloc' might be different between a multithreaded build and a non-multithreaded build. There is no supported way to combine multithreaded code and code that was not compiled to be multithreaded. It may happen to work, but that's a lousy way to make security-sensitive software. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED] :§I"Ï®ÞrØm¶ÿà (¥éì²Z+7¯zZ)éí1¨¥x Ëh¥éì²W^¾^Ë%¢¸ ºÚ&j×.+-1©Úêæj:+v¨¢§²Éh®
RE: Static global - bug? (Re: Two valgrind warnings in OpenSSL - possible bug???)
> I should be able to create a multithreaded application using > a non-multithreaded openssl build provided that I have an ssl > context per thread. Most definitely not. At a minimum, the definition of things like 'errno' and 'malloc' might be different between a multithreaded build and a non-multithreaded build. There is no supported way to combine multithreaded code and code that was not compiled to be multithreaded. It may happen to work, but that's a lousy way to make security-sensitive software. DS __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager [EMAIL PROTECTED]
