In trying out an OpenSSL 3.1 build with FIPS enabled I realized that our cryptohash code had a small issue. Calling a banned cipher generated two different error messages interleaved:
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: unsupported
postgres=# select md5('foo');
ERROR: could not compute MD5 hash: initialization error
It turns out that OpenSSL places two errors in the queue for this operation,
and we only consume one without clearing the queue in between, so we grab an
error from the previous run.
Consuming all (both) errors and creating a concatenated string seems overkill
as it would alter the API from a const error string to something that needs
freeing etc (also, very few OpenSSL consumers actually drain the queue, OpenSSL
themselves don't). Skimming the OpenSSL code I was unable to find another
example of two errors generated. The attached calls ERR_clear_error() as how
we do in libpq in order to avoid consuming earlier errors.
--
Daniel Gustafsson https://vmware.com/
v1-0001-Clear-the-OpenSSL-error-queue-before-cryptohash-o.patch
Description: Binary data
