On 8/15/13 10:24 AM, Nico Williams wrote:
. Recent developments, like Android's failure to properly initialize
OpenSSL's PRNG make me think it's time to table (in the British sense)
the issue once more.
Can you point to any article or post which explains exactly what the
OpenSSL half of the Android issue was? (I understand the Harmony
SecureRandom issue, but that's a separate thing.) OpenSSL is supposed
to call RAND_poll on the first call to RAND_bytes, and RAND_poll knows
how to seed from /dev/urandom on systems that have it, which should
include Android. Neither I nor others speculating on Google+ could
figure out why this wasn't the case, and why explicit seeding would have
been necessary:
https://plus.google.com/+AndroidDevelopers/posts/YxWzeNQMJS2
There should be no need for run-time, global initialization of
OpenSSL (or any sub-system of it) by applications. Moreover, any
existing functions for such initialization should become no-ops or do
their work just once, and in a thread-safe way. As much configuration
of threading and other system libraries should be done at build-time
as possible; as little should be done at run-time as possible. Even
for special-purpose OSes.
Figuring out the right sequence of initialization functions to call,
even when just one application is using OpenSSL, has not been entirely
clear. In particular, see my rambling talk-page discourse on the
OpenSSL Wiki about what is and isn't necessary in order to get ENGINEs
initialized, and how it depends upon some build-time #defines in a
non-obvious way:
http://wiki.opensslfoundation.com/index.php/Talk:Libcrypto_API
OpenSSL requires too much explicit and global run-time initialization
of it by applications in ways that preclude (or at least used to and
still might cause problems for) multiple distinct callers of OpenSSL
in the same process. And yet the layered software systems we've built
over the past decade result in just that happening.
Thank you! I'm glad I'm not the only one who feels this is a big
problem. It's something I've expressed concern about in the past,
albeit in a parenthesized paragraph about halfway through a long,
far-reaching rant:
http://lists.randombit.net/pipermail/cryptography/2012-October/003388.html
Typical examples
include: applications that use TLS and Kerberos, PAM applications
(each module might need to use OpenSSL, and so might the application),
systems running without nscd (the name service switch modules may need
OpenSSL, and so might the app), and so on.
Some examples close to my heart:
* using the OpenSSL Ruby Gem, while also using another Ruby Gem that
depends on OpenSSL indirectly
* using OpenSSL directly while also using libevent's optional
integration with OpenSSL
Huh? Which is it? Must apps call CRYPTO_THREADID_set_callback() even
though "[o]n Windows and many Unix systems, OpenSSL automatically uses
the multi-threaded versions of the standard libraries"? Why? One
would think that the "multi-threaded versions of the standard
libraries" on such OSes would provide all that OpenSSL needs. I don't
get it.
I think what it means is that "We link against thread-safe versions of
the standard library (which implies there are thread-unsafe versions of
the standard library, which I think might be true on Windows, but I'm
pretty sure there isn't any thread-unsafe version of the standard
library on Linux or OS X) so that, for example, errno doesn't get
clobbered by multiple threads. But we still don't call any threading
functions to make OpenSSL itself threadsafe, so you'll have to provide
this boilerplate yourself in every OpenSSL application you write."
I could also rant about OpenSSL's "homemade" thread local storage (TLS,
in the other sense of the acronym) implementation, rather than using the
OS's API for thread local storage. This can result in memory leaks when
a thread terminates, unless you have complete control over the lifecycle
of every thread and can call a cleanup function before the thread
terminates. (And, as you've noted, in a modern application with many
layers of libraries, you don't have such control.)
Finally, what can I do to help?
Definitely to the extent that you can figure out what the actual
behavior is (although that itself is not easy and usually requires
experimentation and studying the source code in-depth), document your
findings on the OpenSSL wiki:
http://wiki.opensslfoundation.com/
(Though hopefully any such clarifications can be propagated to the "more
official" documentation as well.)
And I certainly hope that the initialization and thread-safety issues
can be addressed in libcrypto itself, but if they can't or won't, then I
think we (meaning those interested) should write a small add-on library
to go along with OpenSSL, that would include "standard" implementations
of the thread safety callbacks, and perhaps we could convince users of
OpenSSL to accept it as a de-facto standard, to avoid some of the
"multiple libraries trying to set their own thread callbacks" issues.
(I've actually been meaning to write such an "unofficial companion
library for OpenSSL" for a while, but haven't gotten around to it yet.)
Still though, I'd love it if it could be solved officially and we didn't
need an unofficial companion library.
What fixes might be welcomed? Would
patches to automatically initialize the PRNG on first use (and so on)
be welcomed?
Again, doesn't it *already* do this? In what cases does it not, and how
did Android get tripped up by this?
--Patrick
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
