On 8/30/2020 19:28, Viktor Dukhovni wrote:
On Sun, Aug 30, 2020 at 05:45:41PM -0500, David Arnold wrote:If you prefer this mailing list over github issues, I still want to ask for comments on: Certificate hot-reloading #12753 <https://github.com/openssl/openssl/issues/12753> Specifically, my impression is that this topic has died down a bit and from the linked mailing list threads, in my eye, no concrete conclusion was drawn. I'm not sure how to rank this motion in the context of OpenSSL development, but I guess OpenSSL is used to producing ripple effects, so the man-hour argument might be a genuinely valid one. Please inform my research about this issue with your comments!This is a worthwhile topic. It has a few interesting aspects: 1. Automatic key+cert reloads upon updates of key+cert chain PEM files. This can be tricky when processes start privileged, load the certs and then drop privs, and are no longer able to reopen the key + cert chain file. - Here, for POSIX systems I'd go with an approach where it is the containing directory that is restricted to root or similar, and the actual cert files are group and or world readable. The process can then keep the directory file descriptor open, and then openat(2) to periodically check the cert file, reloading when the metadata changes. - With non-POSIX systems, or applications that don't drop privs, the openat(2) is not needed, and one just checks the cert chain periodically. - Another option is to use passphrase-protected keys, and load the secret passphrase at process start from a separate read-protected file, while the actual private key + cert chain file is world readable, with the access control via protecting the passphrase file. - In all cases, it is important to keep both the private key and the cert in the same file, and open it just once to read both, avoiding races in which the key and cert are read in a way that results in one or the other being stale. 2. Having somehow obtained a new key + cert chain, one now wants to non-disruptively apply them to running servers. Here there are two potential approaches: - Hot plug a new pointer into an existing SSL_CTX structure. While the update itself could be made atomic, the readers of such pointers might read them more than once to separately extract the key and the cert chain, without checking that they're using the same pointer for both operations. This is bound to be fragile, though not necessarily impossible. - Build a new SSL_CTX, and use it to accept *new* connections, while existing connections use whatever SSL_CTX they started with. I believe this can work well, because "SSL" handles increment the reference count of the associated SSL_CTX when they're created, and decrement it when destroyed. So when you create a replacement SSL_CTX, you can just SSL_CTX_free() the old, and it will only actually be deleted when the last SSL connection tied to that SSL_CTX is destroyed. It is true that typical SSL_CTX construction is modestly expensive (loading CA stores and the like) but some of that could be handled by sharing and reference-counting the stores. So my preferred approach would be to create a new SSL_CTX, and get new connections using that. Now in a multi-threaded server, it could be a bit tricky to ensure that the SSL_CTX_free() does not happen before all threads reading the pointer to the latest SSL_CTX see the new pointer installed. Something equivalent to RCU may be needed to ensure that the free only happens after the new pointer is visible in all threads. Designs addressing various parts of this would be cool, provided they're well thought out, and not just single-use-case quick hacks.
This works now; I use it with an application that checks in with a license server and can grab a new cert. OpenSSL appears to have no problem with setting up a new SSL_CTX and using it for new connections; the old ones continue onward until they terminate, and new ones are fine as well.
This appears to be be ok with the current code; I've yet to have it blow up in my face although at present the certs in question are reasonably long-lived. Whether it's robust enough to handle very short-term certificates I do not know.
-- Karl Denninger k...@denninger.net <mailto:k...@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/
smime.p7s
Description: S/MIME Cryptographic Signature
