On Jun 24, 2010, at 4:58 PM, Ger Hobbelt via RT wrote: > The only critical bit here is opinion replacing analysis.
I won't get into a personal attack. > As you said so yourself, and I quote: "When both an application and a > library ([...]) both initialize and register their own threading functions, > bad things happen and the application can crash.") > Indeed. Spot on, I'd say. > > That is /exactly/ why OpenSSL places the ability, and hence also the > responsibility [which is the evil siamese twin of 'ability'], to _provide_ > an singular set of locking operations to the OpenSSL library itself for its > use. With all due respect, OpenSSL is being used as a core/standard library on Linux, the BSD's, and to a lesser extent on the commercial UNIX's. Every standard library has built-in support for multi-threading, and depending on the library that support may come via a series of platform-specific abstraction modules. An application might need to provide its own platform-specific modules, but never for the standard libraries it uses. > ... > Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's > /environment/, either other libraries using it, where the designers may have > missed this spot (read: they don't 'propagate' threading/locking primitives > specification to the next upper design block level) or the application > [programmer] itself, who'd rather not have the resposibility, so he can > spare his remaining synapses for other use. In doing so, they assume a lot, > including OpenSSL being there for them specifically, while OpenSSL is a > library meant (and designed) for a plethora of environments, including quite > a few that definitely do sport requirements agonizingly perpendicular to > yours. Having struggled over the years to get OpenSSL (and CUPS, for that matter) running on diverse platforms, and having done a LOT of bare metal development early in my career, I can appreciate wanting to keep any system-level interfaces out of a toolkit and keeping the code as simple as possible. That said, OpenSSL already exposes POSIX concepts like file descriptors and filenames, even though many platforms do not use file descriptors or filenames natively at all. At some level you need to decide: do you want OpenSSL to be a low-level toolkit that can't be used by applications directly, or do you want it to be a standard library where basic, core functionality is implemented by the library and not by the calling application? > I have (had to) port several other libraries to embedded/non-UNIX > environments (most of those libs originating with the GNU/BSD/Linux > collective and 'of course' carrying all sorts of assumptions that are truish > within that originating environment) and quite a few of them had threading / > locking demands. The only one that was rather plug-and-play was OpenSSL; I've never found the OpenSSL build system to be "plug-and-play"... > I have not had the pleasure to port libcups yet, but I can imagine they > succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I (FWIW, the "they" for CUPS is mostly "me"...) Actually, no. There are certain assumptions that I've made over the years (mainly that there will be a BSD-like sockets interface and 32-bit or larger ints), but overall the code has proven very portable and I have carefully abstracted problematic interfaces (like encryption, file IO, memory allocation, threading, and mutexes) so that porting is often just a matter of providing a new implementation of those interfaces specific to the platform. Of course, by doing so I've made libcups marginally larger and harder to maintain. However, all of the applications that use libcups know that they don't need to bother with OS-specific stuff unless they want to, and I'd hazard a guess that the maintenance is far less involved than, say, keeping a lot of hand-optimized assembly for platform X working. > ... > Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a > couple of lines more code to write then when 'they' (for any amount of > 'they') had provided you with a ready-to-go implementation custom tailored > for your rig, but that 'lack' can be quickly alleviated by looking for a bit > of sample code, some judicious copy&paste&expand-on-that and you're good to > go. Besides, you get to have the wonderful sense of accomplishment that day. > Which is a valuable benefit, considering the usual paycheck. I find no benefit in re-implementing the same code multiple times, particularly when my implementation will likely conflict with the next developer's implementation (even if it is the same code!) I think it is great that you have abstracted all of the locking interfaces within OpenSSL. I took a similar approach in CUPS and other libraries I have done. However, I also provided implementations for the popular platforms (that I was willing to support directly) in those projects themselves to provide the greatest benefits - people can still port my projects to new platforms but they only have to implement/instantiate the platform-specific code *once* for all of the applications that use that project instead of N times. Moreover, by putting that platform code in the project I avoid the side-effects we see with OpenSSL usage on Linux, usage which is predictable when a library becomes a standard part of an OS and is used by multiple, separately-maintained bits of code. > ... > On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT > <[email protected]>wrote: > >> An emerging issue with CUPS and other applications that use it has >> identified a critical design flaw in how OpenSSL supports multithreading. >> >> Specifically, a multi-threaded application or library that uses OpenSSL >> must provide its own threading functions for OpenSSL to use. When both an >> application and a library (say, Firefox and libcups) both initialize and >> register their own threading functions, bad things happen and the >> application can crash. >> >> Pushing the responsibility of making OpenSSL thread-safe on the application >> is a bad design choice and needs to be fixed ASAP to allow cross-platform, >> multi-threaded applications to be developed safely. >> >> ________________________________________________________________________ >> Michael Sweet, Senior Printing System Engineer, PWG Chair >> >> >> >> >> ______________________________________________________________________ >> OpenSSL Project http://www.openssl.org >> Development Mailing List [email protected] >> Automated List Manager [email protected] >> > > > > -- > Met vriendelijke groeten / Best regards, > > Ger Hobbelt > > -------------------------------------------------- > web: http://www.hobbelt.com/ > http://www.hebbut.net/ > mail: [email protected] > mobile: +31-6-11 120 978 > -------------------------------------------------- > > The only critical bit here is opinion replacing analysis. > > As you said so yourself, and I quote: "When both an application and a library > ([...]) both initialize and register their own threading functions, bad > things happen and the application can crash.") > Indeed. Spot on, I'd say. > > That is /exactly/ why OpenSSL places the ability, and hence also the > responsibility [which is the evil siamese twin of 'ability'], to _provide_ an > singular set of locking operations to the OpenSSL library itself for its use. > These are > CRYPTO_set_lock_callback > CRYPTO_add_locking_callback > CRYPTO_set_dynlock_create_callback > CRYPTO_set_dynlock_lock_callback > CRYPTO_set_dynlock_destroy_callback > and you must call those at library setup time, say, around the same time you > invoke ERR_load_crypto_strings(). > > > [The next bit possibly is not going to adhere to soothingly gentle didactic > doctrine, but I'm old enough to still appreciate the sentiment of Proverbs > 13:24 at times.] > > To put not to fine a point on it, I wonder how I could have missed such a > critical issue for the last 11 years, while I've quite successfully used > OpenSSL in (and ported the library itself to!) several environments, > including embedded environments (and about 'embedded': I consider only those > environments 'embedded' which are not simply a Linux/UNIX port/emulation - > those are for tenderfoots. ;-) <evil smirk> In a truly embedded > environment, you /don't/ have the luxury of stdin/stdout, let alone stderr, > to name but one. The only standard thing there is you ain't got POSIX nor any > ISO-ish 'standard' run-time library, to name another. Either you get it right > the first time so your serial connection or network connection actually > works, or you're down to hack a kind of POST code with the few LEDs you've > got (if any), or you're hopefully going to be that lucky s.o.b. with a cuddly > budget who's debugging using some fancy JTAG hardware - or an ICE if you get > to be _really_ posh. > > > Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's > /environment/, either other libraries using it, where the designers may have > missed this spot (read: they don't 'propagate' threading/locking primitives > specification to the next upper design block level) or the application > [programmer] itself, who'd rather not have the resposibility, so he can spare > his remaining synapses for other use. In doing so, they assume a lot, > including OpenSSL being there for them specifically, while OpenSSL is a > library meant (and designed) for a plethora of environments, including quite > a few that definitely do sport requirements agonizingly perpendicular to > yours. > > I have (had to) port several other libraries to embedded/non-UNIX > environments (most of those libs originating with the GNU/BSD/Linux > collective and 'of course' carrying all sorts of assumptions that are truish > within that originating environment) and quite a few of them had threading / > locking demands. The only one that was rather plug-and-play was OpenSSL; > those others all had threading/locking support 'built-in' to make app devs > happy and I had to forcibly kick that code out of there (and substitute it, > of course) before I'd even have a chance to get them to work in my > environments - given the permeativeness of those elemental bits throughout > the code, I can only say doing that sort of 'portability prep work' can do > wonders for my temper. > > I have not had the pleasure to port libcups yet, but I can imagine they > succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I > think) and you run into those assumptions there, got fazed, fizzed, and the > next grep around the corner landing in OpenSSL County delivered a Blame > Pointer right when you needed one. No hard feelings, but this is not a bug, > but the difference between a multi-platform library and library/application > code which assumes a certain predefined platform set from the get-go. > > Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a > couple of lines more code to write then when 'they' (for any amount of > 'they') had provided you with a ready-to-go implementation custom tailored > for your rig, but that 'lack' can be quickly alleviated by looking for a bit > of sample code, some judicious copy&paste&expand-on-that and you're good to > go. Besides, you get to have the wonderful sense of accomplishment that day. > Which is a valuable benefit, considering the usual paycheck. > > > > > Note/By The Way: > When you see the call CRYPTO_thread_setup() invoked somewhere, /anywhere/, > then you've got your hands on a piece of code which uses severely antiquated > support code from OpenSSL (the fossil is still buried in > crypto/threads/th-lock.c. When you feel like it, compare that code with the > list of methods specified at top and arrive to your own conclusion there. > Before anyone feels like piping up right after that happens: do another grep > to verify whether that code is 'visible' outside OpenSSL (hint: prototypes > don't feature in any header file; for good reason)) > > > > On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT <[email protected]> > wrote: > An emerging issue with CUPS and other applications that use it has identified > a critical design flaw in how OpenSSL supports multithreading. > > Specifically, a multi-threaded application or library that uses OpenSSL must > provide its own threading functions for OpenSSL to use. When both an > application and a library (say, Firefox and libcups) both initialize and > register their own threading functions, bad things happen and the application > can crash. > > Pushing the responsibility of making OpenSSL thread-safe on the application > is a bad design choice and needs to be fixed ASAP to allow cross-platform, > multi-threaded applications to be developed safely. > > ________________________________________________________________________ > Michael Sweet, Senior Printing System Engineer, PWG Chair > > > > > ______________________________________________________________________ > OpenSSL Project http://www.openssl.org > Development Mailing List [email protected] > Automated List Manager [email protected] > > > > -- > Met vriendelijke groeten / Best regards, > > Ger Hobbelt > > -------------------------------------------------- > web: http://www.hobbelt.com/ > http://www.hebbut.net/ > mail: [email protected] > mobile: +31-6-11 120 978 > -------------------------------------------------- > ________________________________________________________________________ Michael Sweet, Senior Printing System Engineer, PWG Chair
On Jun 24, 2010, at 4:58 PM, Ger Hobbelt via RT wrote: The only critical bit here is opinion replacing analysis. I won't get into a personal attack. As you said so yourself, and I quote: "When both an application and a
... That said, OpenSSL already exposes POSIX concepts like file descriptors and filenames, even though many platforms do not use file descriptors or filenames natively at all. At some level you need to decide: do you want OpenSSL to be a low-level toolkit that can't be used by applications directly, or do you want it to be a standard library where basic, core functionality is implemented by the library and not by the calling application? I have (had to) port several other libraries to embedded/non-UNIX I have not had the pleasure to port libcups yet, but I can imagine they succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I (FWIW, the "they" for CUPS is mostly "me"...) Of course, by doing so I've made libcups marginally larger and harder to maintain. However, all of the applications that use libcups know that they don't need to bother with OS-specific stuff unless they want to, and I'd hazard a guess that the maintenance is far less involved than, say, keeping a lot of hand-optimized assembly for platform X working. ... Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a I find no benefit in re-implementing the same code multiple times, particularly when my implementation will likely conflict with the next developer's implementation (even if it is the same code!) I think it is great that you have abstracted all of the locking interfaces within OpenSSL. I took a similar approach in CUPS and other libraries I have done. However, I also provided implementations for the popular platforms (that I was willing to support directly) in those projects themselves to provide the greatest benefits - people can still port my projects to new platforms but they only have to implement/instantiate the platform-specific code *once* for all of the applications that use that project instead of N times. Moreover, by putting that platform code in the project I avoid the side-effects we see with OpenSSL usage on Linux, usage which is predictable when a library becomes a standard part of an OS and is used by multiple, separately-maintained bits of code. ... ________________________________________________________________________ Michael Sweet, Senior Printing System Engineer, PWG Chair |
