On Jun 24, 2010, at 4:58 PM, Ger Hobbelt via RT wrote:
> The only critical bit here is opinion replacing analysis.

I won't get into a personal attack.

> As you said so yourself, and I quote: "When both an application and a
> library ([...]) both initialize and register their own threading functions,
> bad things happen and the application can crash.")
> Indeed. Spot on, I'd say.
> 
> That is /exactly/ why OpenSSL places the ability, and hence also the
> responsibility [which is the evil siamese twin of 'ability'], to _provide_
> an singular set of locking operations to the OpenSSL library itself for its
> use.

With all due respect, OpenSSL is being used as a core/standard library on 
Linux, the BSD's, and to a lesser extent on the commercial UNIX's. Every 
standard library has built-in support for multi-threading, and depending on the 
library that support may come via a series of platform-specific abstraction 
modules.  An application might need to provide its own platform-specific 
modules, but never for the standard libraries it uses.

> ...
> Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's
> /environment/, either other libraries using it, where the designers may have
> missed this spot (read: they don't 'propagate' threading/locking primitives
> specification to the next upper design block level) or the application
> [programmer] itself, who'd rather not have the resposibility, so he can
> spare his remaining synapses for other use. In doing so, they assume a lot,
> including OpenSSL being there for them specifically, while OpenSSL is a
> library meant (and designed) for a plethora of environments, including quite
> a few that definitely do sport requirements agonizingly perpendicular to
> yours.

Having struggled over the years to get OpenSSL (and CUPS, for that matter) 
running on diverse platforms, and having done a LOT of bare metal development 
early in my career, I can appreciate wanting to keep any system-level 
interfaces out of a toolkit and keeping the code as simple as possible.

That said, OpenSSL already exposes POSIX concepts like file descriptors and 
filenames, even though many platforms do not use file descriptors or filenames 
natively at all. At some level you need to decide: do you want OpenSSL to be a 
low-level toolkit that can't be used by applications directly, or do you want 
it to be a standard library where basic, core functionality is implemented by 
the library and not by the calling application?

> I have (had to) port several other libraries to embedded/non-UNIX
> environments (most of those libs originating with the GNU/BSD/Linux
> collective and 'of course' carrying all sorts of assumptions that are truish
> within that originating environment) and quite a few of them had threading /
> locking demands. The only one that was rather plug-and-play was OpenSSL;

I've never found the OpenSSL build system to be "plug-and-play"...

> I have not had the pleasure to port libcups yet, but I can imagine they
> succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I

(FWIW, the "they" for CUPS is mostly "me"...)

Actually, no.  There are certain assumptions that I've made over the years 
(mainly that there will be a BSD-like sockets interface and 32-bit or larger 
ints), but overall the code has proven very portable and I have carefully 
abstracted problematic interfaces (like encryption, file IO, memory allocation, 
threading, and mutexes) so that porting is often just a matter of providing a 
new implementation of those interfaces specific to the platform.

Of course, by doing so I've made libcups marginally larger and harder to 
maintain. However, all of the applications that use libcups know that they 
don't need to bother with OS-specific stuff unless they want to, and I'd hazard 
a guess that the maintenance is far less involved than, say, keeping a lot of 
hand-optimized assembly for platform X working.

> ...
> Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a
> couple of lines more code to write then when 'they' (for any amount of
> 'they') had provided you with a ready-to-go implementation custom tailored
> for your rig, but that 'lack' can be quickly alleviated by looking for a bit
> of sample code, some judicious copy&paste&expand-on-that and you're good to
> go. Besides, you get to have the wonderful sense of accomplishment that day.
> Which is a valuable benefit, considering the usual paycheck.

I find no benefit in re-implementing the same code multiple times, particularly 
when my implementation will likely conflict with the next developer's 
implementation (even if it is the same code!)

I think it is great that you have abstracted all of the locking interfaces 
within OpenSSL. I took a similar approach in CUPS and other libraries I have 
done. However, I also provided implementations for the popular platforms (that 
I was willing to support directly) in those projects themselves to provide the 
greatest benefits - people can still port my projects to new platforms but they 
only have to implement/instantiate the platform-specific code *once* for all of 
the applications that use that project instead of N times.  Moreover, by 
putting that platform code in the project I avoid the side-effects we see with 
OpenSSL usage on Linux, usage which is predictable when a library becomes a 
standard part of an OS and is used by multiple, separately-maintained bits of 
code.

> ...
> On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT 
> <[email protected]>wrote:
> 
>> An emerging issue with CUPS and other applications that use it has
>> identified a critical design flaw in how OpenSSL supports multithreading.
>> 
>> Specifically, a multi-threaded application or library that uses OpenSSL
>> must provide its own threading functions for OpenSSL to use.  When both an
>> application and a library (say, Firefox and libcups) both initialize and
>> register their own threading functions, bad things happen and the
>> application can crash.
>> 
>> Pushing the responsibility of making OpenSSL thread-safe on the application
>> is a bad design choice and needs to be fixed ASAP to allow cross-platform,
>> multi-threaded applications to be developed safely.
>> 
>> ________________________________________________________________________
>> Michael Sweet, Senior Printing System Engineer, PWG Chair
>> 
>> 
>> 
>> 
>> ______________________________________________________________________
>> OpenSSL Project                                 http://www.openssl.org
>> Development Mailing List                       [email protected]
>> Automated List Manager                           [email protected]
>> 
> 
> 
> 
> -- 
> Met vriendelijke groeten / Best regards,
> 
> Ger Hobbelt
> 
> --------------------------------------------------
> web:    http://www.hobbelt.com/
>       http://www.hebbut.net/
> mail:   [email protected]
> mobile: +31-6-11 120 978
> --------------------------------------------------
> 
> The only critical bit here is opinion replacing analysis.
> 
> As you said so yourself, and I quote: "When both an application and a library 
> ([...]) both initialize and register their own threading functions, bad 
> things happen and the application can crash.")
> Indeed. Spot on, I'd say.
> 
> That is /exactly/ why OpenSSL places the ability, and hence also the 
> responsibility [which is the evil siamese twin of 'ability'], to _provide_ an 
> singular set of locking operations to the OpenSSL library itself for its use.
> These are
> CRYPTO_set_lock_callback
> CRYPTO_add_locking_callback
> CRYPTO_set_dynlock_create_callback
> CRYPTO_set_dynlock_lock_callback
> CRYPTO_set_dynlock_destroy_callback
> and you must call those at library setup time, say, around the same time you 
> invoke ERR_load_crypto_strings().
> 
> 
> [The next bit possibly is not going to adhere to soothingly gentle didactic 
> doctrine, but I'm old enough to still appreciate the sentiment of Proverbs 
> 13:24 at times.]
> 
> To put not to fine a point on it, I wonder how I could have missed such a 
> critical issue for the last 11 years, while I've quite successfully used 
> OpenSSL in (and ported the library itself to!) several environments, 
> including embedded environments (and about 'embedded': I consider only those 
> environments 'embedded' which are not simply a Linux/UNIX port/emulation - 
> those are for tenderfoots. ;-) <evil smirk>   In a truly embedded 
> environment, you /don't/ have the luxury of stdin/stdout, let alone stderr, 
> to name but one. The only standard thing there is you ain't got POSIX nor any 
> ISO-ish 'standard' run-time library, to name another. Either you get it right 
> the first time so your serial connection or network connection actually 
> works, or you're down to hack a kind of POST code with the few LEDs you've 
> got (if any), or you're hopefully going to be that lucky s.o.b. with a cuddly 
> budget who's debugging using some fancy JTAG hardware - or an ICE if you get 
> to be _really_ posh.
> 
> 
> Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's 
> /environment/, either other libraries using it, where the designers may have 
> missed this spot (read: they don't 'propagate' threading/locking primitives 
> specification to the next upper design block level) or the application 
> [programmer] itself, who'd rather not have the resposibility, so he can spare 
> his remaining synapses for other use. In doing so, they assume a lot, 
> including OpenSSL being there for them specifically, while OpenSSL is a 
> library meant (and designed) for a plethora of environments, including quite 
> a few that definitely do sport requirements agonizingly perpendicular to 
> yours.
> 
> I have (had to) port several other libraries to embedded/non-UNIX 
> environments (most of those libs originating with the GNU/BSD/Linux 
> collective and 'of course' carrying all sorts of assumptions that are truish 
> within that originating environment) and quite a few of them had threading / 
> locking demands. The only one that was rather plug-and-play was OpenSSL; 
> those others all had threading/locking support 'built-in' to make app devs 
> happy and I had to forcibly kick that code out of there (and substitute it, 
> of course) before I'd even have a chance to get them to work in my 
> environments - given the permeativeness of those elemental bits throughout 
> the code, I can only say doing that sort of 'portability prep work' can do 
> wonders for my temper.
> 
> I have not had the pleasure to port libcups yet, but I can imagine they 
> succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I 
> think) and you run into those assumptions there, got fazed, fizzed, and the 
> next grep around the corner landing in OpenSSL County delivered a Blame 
> Pointer right when you needed one. No hard feelings, but this is not a bug, 
> but the difference between a multi-platform library and library/application 
> code which assumes a certain predefined platform set from the get-go.
> 
> Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a 
> couple of lines more code to write then when 'they' (for any amount of 
> 'they') had provided you with a ready-to-go implementation custom tailored 
> for your rig, but that 'lack' can be quickly alleviated by looking for a bit 
> of sample code, some judicious copy&paste&expand-on-that and you're good to 
> go. Besides, you get to have the wonderful sense of accomplishment that day. 
> Which is a valuable benefit, considering the usual paycheck.
> 
> 
> 
> 
> Note/By The Way:
> When you see the call CRYPTO_thread_setup() invoked somewhere, /anywhere/, 
> then you've got your hands on a piece of code which uses severely antiquated 
> support code from OpenSSL (the fossil is still buried in 
> crypto/threads/th-lock.c. When you feel like it, compare that code with the 
> list of methods specified at top and arrive to your own conclusion there. 
> Before anyone feels like piping up right after that happens: do another grep 
> to verify whether that code is 'visible' outside OpenSSL (hint: prototypes 
> don't feature in any header file; for good reason))
> 
> 
> 
> On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT <[email protected]> 
> wrote:
> An emerging issue with CUPS and other applications that use it has identified 
> a critical design flaw in how OpenSSL supports multithreading.
> 
> Specifically, a multi-threaded application or library that uses OpenSSL must 
> provide its own threading functions for OpenSSL to use.  When both an 
> application and a library (say, Firefox and libcups) both initialize and 
> register their own threading functions, bad things happen and the application 
> can crash.
> 
> Pushing the responsibility of making OpenSSL thread-safe on the application 
> is a bad design choice and needs to be fixed ASAP to allow cross-platform, 
> multi-threaded applications to be developed safely.
> 
> ________________________________________________________________________
> Michael Sweet, Senior Printing System Engineer, PWG Chair
> 
> 
> 
> 
> ______________________________________________________________________
> OpenSSL Project                                 http://www.openssl.org
> Development Mailing List                       [email protected]
> Automated List Manager                           [email protected]
> 
> 
> 
> -- 
> Met vriendelijke groeten / Best regards,
> 
> Ger Hobbelt
> 
> --------------------------------------------------
> web:    http://www.hobbelt.com/
>        http://www.hebbut.net/
> mail:   [email protected]
> mobile: +31-6-11 120 978
> --------------------------------------------------
> 

________________________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair





On Jun 24, 2010, at 4:58 PM, Ger Hobbelt via RT wrote:
The only critical bit here is opinion replacing analysis.

I won't get into a personal attack.

As you said so yourself, and I quote: "When both an application and a
library ([...]) both initialize and register their own threading functions,
bad things happen and the application can crash.")
Indeed. Spot on, I'd say.

That is /exactly/ why OpenSSL places the ability, and hence also the
responsibility [which is the evil siamese twin of 'ability'], to _provide_
an singular set of locking operations to the OpenSSL library itself for its
use.

With all due respect, OpenSSL is being used as a core/standard library on Linux, the BSD's, and to a lesser extent on the commercial UNIX's. Every standard library has built-in support for multi-threading, and depending on the library that support may come via a series of platform-specific abstraction modules.  An application might need to provide its own platform-specific modules, but never for the standard libraries it uses.

...
Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's
/environment/, either other libraries using it, where the designers may have
missed this spot (read: they don't 'propagate' threading/locking primitives
specification to the next upper design block level) or the application
[programmer] itself, who'd rather not have the resposibility, so he can
spare his remaining synapses for other use. In doing so, they assume a lot,
including OpenSSL being there for them specifically, while OpenSSL is a
library meant (and designed) for a plethora of environments, including quite
a few that definitely do sport requirements agonizingly perpendicular to
yours.

Having struggled over the years to get OpenSSL (and CUPS, for that matter) running on diverse platforms, and having done a LOT of bare metal development early in my career, I can appreciate wanting to keep any system-level interfaces out of a toolkit and keeping the code as simple as possible.

That said, OpenSSL already exposes POSIX concepts like file descriptors and filenames, even though many platforms do not use file descriptors or filenames natively at all. At some level you need to decide: do you want OpenSSL to be a low-level toolkit that can't be used by applications directly, or do you want it to be a standard library where basic, core functionality is implemented by the library and not by the calling application?

I have (had to) port several other libraries to embedded/non-UNIX
environments (most of those libs originating with the GNU/BSD/Linux
collective and 'of course' carrying all sorts of assumptions that are truish
within that originating environment) and quite a few of them had threading /
locking demands. The only one that was rather plug-and-play was OpenSSL;

I've never found the OpenSSL build system to be "plug-and-play"...

I have not had the pleasure to port libcups yet, but I can imagine they
succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I

(FWIW, the "they" for CUPS is mostly "me"...)

Actually, no.  There are certain assumptions that I've made over the years (mainly that there will be a BSD-like sockets interface and 32-bit or larger ints), but overall the code has proven very portable and I have carefully abstracted problematic interfaces (like encryption, file IO, memory allocation, threading, and mutexes) so that porting is often just a matter of providing a new implementation of those interfaces specific to the platform.

Of course, by doing so I've made libcups marginally larger and harder to maintain. However, all of the applications that use libcups know that they don't need to bother with OS-specific stuff unless they want to, and I'd hazard a guess that the maintenance is far less involved than, say, keeping a lot of hand-optimized assembly for platform X working.

...
Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a
couple of lines more code to write then when 'they' (for any amount of
'they') had provided you with a ready-to-go implementation custom tailored
for your rig, but that 'lack' can be quickly alleviated by looking for a bit
of sample code, some judicious copy&paste&expand-on-that and you're good to
go. Besides, you get to have the wonderful sense of accomplishment that day.
Which is a valuable benefit, considering the usual paycheck.

I find no benefit in re-implementing the same code multiple times, particularly when my implementation will likely conflict with the next developer's implementation (even if it is the same code!)

I think it is great that you have abstracted all of the locking interfaces within OpenSSL. I took a similar approach in CUPS and other libraries I have done. However, I also provided implementations for the popular platforms (that I was willing to support directly) in those projects themselves to provide the greatest benefits - people can still port my projects to new platforms but they only have to implement/instantiate the platform-specific code *once* for all of the applications that use that project instead of N times.  Moreover, by putting that platform code in the project I avoid the side-effects we see with OpenSSL usage on Linux, usage which is predictable when a library becomes a standard part of an OS and is used by multiple, separately-maintained bits of code.

...
On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT <[email protected]>wrote:

An emerging issue with CUPS and other applications that use it has
identified a critical design flaw in how OpenSSL supports multithreading.

Specifically, a multi-threaded application or library that uses OpenSSL
must provide its own threading functions for OpenSSL to use.  When both an
application and a library (say, Firefox and libcups) both initialize and
register their own threading functions, bad things happen and the
application can crash.

Pushing the responsibility of making OpenSSL thread-safe on the application
is a bad design choice and needs to be fixed ASAP to allow cross-platform,
multi-threaded applications to be developed safely.

________________________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair




______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [email protected]
Automated List Manager                           [email protected]




--
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web:    http://www.hobbelt.com/
      http://www.hebbut.net/
mail:   [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

The only critical bit here is opinion replacing analysis.

As you said so yourself, and I quote: "When both an application and a library ([...]) both initialize and register their own threading functions, bad things happen and the application can crash.")
Indeed. Spot on, I'd say.

That is /exactly/ why OpenSSL places the ability, and hence also the responsibility [which is the evil siamese twin of 'ability'], to _provide_ an singular set of locking operations to the OpenSSL library itself for its use.
These are
CRYPTO_set_lock_callback
CRYPTO_add_locking_callback
CRYPTO_set_dynlock_create_callback
CRYPTO_set_dynlock_lock_callback
CRYPTO_set_dynlock_destroy_callback
and you must call those at library setup time, say, around the same time you invoke ERR_load_crypto_strings().


[The next bit possibly is not going to adhere to soothingly gentle didactic doctrine, but I'm old enough to still appreciate the sentiment of Proverbs 13:24 at times.]

To put not to fine a point on it, I wonder how I could have missed such a critical issue for the last 11 years, while I've quite successfully used OpenSSL in (and ported the library itself to!) several environments, including embedded environments (and about 'embedded': I consider only those environments 'embedded' which are not simply a Linux/UNIX port/emulation - those are for tenderfoots. ;-) <evil smirk>   In a truly embedded environment, you /don't/ have the luxury of stdin/stdout, let alone stderr, to name but one. The only standard thing there is you ain't got POSIX nor any ISO-ish 'standard' run-time library, to name another. Either you get it right the first time so your serial connection or network connection actually works, or you're down to hack a kind of POST code with the few LEDs you've got (if any), or you're hopefully going to be that lucky s.o.b. with a cuddly budget who's debugging using some fancy JTAG hardware - or an ICE if you get to be _really_ posh.


Ranting aside, the problem, if /any/, is not OpenSSL this time, but it's /environment/, either other libraries using it, where the designers may have missed this spot (read: they don't 'propagate' threading/locking primitives specification to the next upper design block level) or the application [programmer] itself, who'd rather not have the resposibility, so he can spare his remaining synapses for other use. In doing so, they assume a lot, including OpenSSL being there for them specifically, while OpenSSL is a library meant (and designed) for a plethora of environments, including quite a few that definitely do sport requirements agonizingly perpendicular to yours.

I have (had to) port several other libraries to embedded/non-UNIX environments (most of those libs originating with the GNU/BSD/Linux collective and 'of course' carrying all sorts of assumptions that are truish within that originating environment) and quite a few of them had threading / locking demands. The only one that was rather plug-and-play was OpenSSL; those others all had threading/locking support 'built-in' to make app devs happy and I had to forcibly kick that code out of there (and substitute it, of course) before I'd even have a chance to get them to work in my environments - given the permeativeness of those elemental bits throughout the code, I can only say doing that sort of 'portability prep work' can do wonders for my temper.

I have not had the pleasure to port libcups yet, but I can imagine they succumbed to those 'we're on UNIX, guys!' assumptions (for good reason, I think) and you run into those assumptions there, got fazed, fizzed, and the next grep around the corner landing in OpenSSL County delivered a Blame Pointer right when you needed one. No hard feelings, but this is not a bug, but the difference between a multi-platform library and library/application code which assumes a certain predefined platform set from the get-go.

Be glad OpenSSL does provide these callbacks, the way it does. Yes, it's a couple of lines more code to write then when 'they' (for any amount of 'they') had provided you with a ready-to-go implementation custom tailored for your rig, but that 'lack' can be quickly alleviated by looking for a bit of sample code, some judicious copy&paste&expand-on-that and you're good to go. Besides, you get to have the wonderful sense of accomplishment that day. Which is a valuable benefit, considering the usual paycheck.




Note/By The Way:
When you see the call CRYPTO_thread_setup() invoked somewhere, /anywhere/, then you've got your hands on a piece of code which uses severely antiquated support code from OpenSSL (the fossil is still buried in crypto/threads/th-lock.c. When you feel like it, compare that code with the list of methods specified at top and arrive to your own conclusion there. Before anyone feels like piping up right after that happens: do another grep to verify whether that code is 'visible' outside OpenSSL (hint: prototypes don't feature in any header file; for good reason))



On Thu, Jun 24, 2010 at 10:19 PM, Michael Sweet via RT <[email protected]> wrote:
An emerging issue with CUPS and other applications that use it has identified a critical design flaw in how OpenSSL supports multithreading.

Specifically, a multi-threaded application or library that uses OpenSSL must provide its own threading functions for OpenSSL to use.  When both an application and a library (say, Firefox and libcups) both initialize and register their own threading functions, bad things happen and the application can crash.

Pushing the responsibility of making OpenSSL thread-safe on the application is a bad design choice and needs to be fixed ASAP to allow cross-platform, multi-threaded applications to be developed safely.

________________________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair




______________________________________________________________________
OpenSSL Project                                 http://www.openssl.org
Development Mailing List                       [email protected]
Automated List Manager                           [email protected]



--
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web:    http://www.hobbelt.com/
       http://www.hebbut.net/
mail:   [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------


________________________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair




Reply via email to