Fix handling of SCRAM-SHA-256's channel binding with RSA-PSS certificates OpenSSL 1.1.1 and newer versions have added support for RSA-PSS certificates, which requires the use of a specific routine in OpenSSL to determine which hash function to use when compiling it when using channel binding in SCRAM-SHA-256. X509_get_signature_nid(), that is the original routine the channel binding code has relied on, is not able to determine which hash algorithm to use for such certificates. However, X509_get_signature_info(), new to OpenSSL 1.1.1, is able to do it. This commit switches the channel binding logic to rely on X509_get_signature_info() over X509_get_signature_nid(), which would be the choice when building with 1.1.1 or newer.
The error could have been triggered on the client or the server, hence libpq and the backend need to have their related code paths patched. Note that attempting to load an RSA-PSS certificate with OpenSSL 1.1.0 or older leads to a failure due to an unsupported algorithm. The discovery of relying on X509_get_signature_info() comes from Jacob, the tests have been written by Heikki (with few tweaks from me), while I have bundled the whole together while adding the bits needed for MSVC and meson. This issue exists since channel binding exists, so backpatch all the way down. Some tests are added in 15~, triggered if compiling with OpenSSL 1.1.1 or newer, where the certificate and key files can easily be generated for RSA-PSS. Reported-by: Gunnar "Nick" Bluth Author: Jacob Champion, Heikki Linnakangas Discussion: https://postgr.es/m/17760-b6c61e752ec07...@postgresql.org Backpatch-through: 11 Branch ------ REL_11_STABLE Details ------- https://git.postgresql.org/pg/commitdiff/88d606f7cc68aa753868ca92b0e065d77c5915d2 Modified Files -------------- configure | 12 ++++++++++++ configure.in | 2 ++ src/backend/libpq/be-secure-openssl.c | 9 +++++++-- src/include/libpq/libpq-be.h | 2 +- src/include/pg_config.h.in | 3 +++ src/interfaces/libpq/fe-secure-openssl.c | 9 +++++++-- src/interfaces/libpq/libpq-int.h | 2 +- src/tools/msvc/Solution.pm | 9 ++++++++- 8 files changed, 41 insertions(+), 7 deletions(-)
