Public bug reported:
[ Impact ]
libp11 0.4.18 (resolute: 0.4.18-1) registers its RSA EVP_PKEY_METHOD
process-wide
with EVP_PKEY_meth_add0() whenever an RSA private key is loaded through
PKCS11_get_private_key(), i.e. on every key load through the pkcs11 ENGINE
(src/p11_rsa.c:307, introduced by upstream commit 60329e0 "Fix RSA-PSS and
RSA-OAEP via PKCS#11 provider on OpenSSL 3.x", first released in 0.4.18).
On OpenSSL 3.x an application-registered EVP_PKEY_METHOD takes priority over the
provider machinery for all RSA keys in the process. After the first engine key
load, unrelated provider-path RSA code in the same process fails:
EVP_PKEY_CTX_new_from_name(NULL, "RSA", NULL) succeeds but
EVP_PKEY_fromdata_init() fails with
error:03000096:digital envelope routines::operation not supported for
this keytype
This breaks EAP-TLS with TPM-backed client keys on Ubuntu 26.04. wpa_supplicant
loads the Wi-Fi client key through the pkcs11 engine (tpm2-pkcs11 via p11-kit),
and tpm2-tss inside the same process then cannot build the parent key's RSA
public key to start its salted TPM session, so the key can never be used:
ERROR:esys_crypto:src/tss2-esys/esys_crypto_ossl.c:728:iesys_cryptossl_pk_encrypt()
ErrorCode (0x00070001) Could not create rsa key.
ERROR:esys:src/tss2-esys/api/Esys_StartAuthSession.c:113:Esys_StartAuthSession()
Error in async function ErrorCode (0x00070001)
ERROR: Could not start Auth Session with the TPM.
ERROR: Error unsealing wrapping key
PKCS11_get_private_key returned NULL
ENGINE: cannot load private key with id
'pkcs11:token=...;object=...;type=private?pin-value=...'
[error:03000096:digital envelope routines::operation not supported for this
keytype]
TLS: Failed to initialize engine
EAP-TLS: Failed to initialize SSL.
wpa_supplicant 2.11 in resolute can only load PKCS#11 keys through the ENGINE
API
(its pkcs11-provider path is compiled only with OPENSSL_NO_ENGINE), so there is
no
configuration-level workaround. Affected machines cannot join 802.1X/EAP-TLS
networks at all. The same setup works on 24.04.
This is the same failure class as LP: #2163152 on noble (process-global engine
state breaking tpm2-tss), but a different code path: noble's was the backported
ENGINE_set_default_string() workaround; resolute's is the 0.4.18 RSA-PSS/OAEP
feature.
Upstream reported this as OpenSC/libp11#672 and fixed it in PR #675 ("Replace
global RSA methods with per-context callbacks", commits 67f5f5b and e72a201),
released in libp11 0.4.21 (2026-09-07): "Isolated ENGINE EVP_PKEY methods from
unrelated OpenSSL keys". Debian unstable/testing carry 0.4.21-1. Nothing is
pending for resolute; LP: #2171132 only covers merging 0.4.21 into tenacious.
[ Test Plan ]
Reproducible with SoftHSM2 in a clean resolute container; no TPM
required.
1. sudo apt install softhsm2 opensc libengine-pkcs11-openssl build-
essential libssl-dev openssl
2. openssl req -x509 -newkey rsa:2048 -keyout k.pem -out c.pem -days 365 -nodes
-subj "/CN=Test"
softhsm2-util --init-token --free --label test --so-pin 1234 --pin 1234
softhsm2-util --import k.pem --token test --label testkey --id 01 --pin 1234
3. Build the probe from LP: #2163152 (loads a key through the engine, then does
unrelated provider-path RSA work, as tpm2-tss does internally):
cat > probe.c <<'EOF'
#include <stdio.h>
#include <openssl/engine.h>
#include <openssl/evp.h>
int main(int argc, char **argv)
{
if (argc < 2) {
fprintf(stderr, "usage: %s <pkcs11-key-uri>\n", argv[0]);
return 2;
}
ENGINE_load_builtin_engines();
ENGINE *e = ENGINE_by_id("pkcs11");
if (!e || !ENGINE_init(e)) {
fprintf(stderr, "failed to load the pkcs11 engine\n");
return 2;
}
EVP_PKEY *k = ENGINE_load_private_key(e, argv[1], NULL, NULL);
printf("load=%s\n", k ? "OK" : "FAILED");
if (!k)
return 2;
EVP_PKEY_CTX *c = EVP_PKEY_CTX_new_from_name(NULL, "RSA", NULL);
int rc = EVP_PKEY_fromdata_init(c);
printf("fromdata_init=%d -> %s\n", rc, rc > 0 ? "CLEAN" : "POLLUTED");
return rc > 0 ? 0 : 1;
}
EOF
gcc -Wno-deprecated-declarations probe.c -o probe -lcrypto
4. ./probe 'pkcs11:token=test;object=testkey;type=private;pin-
value=1234'
Results on Ubuntu 26.04.1, openssl 3.5.5-1ubuntu3.7:
libengine-pkcs11-openssl 0.4.18-1 (archive) fromdata_init=-2 -> POLLUTED
upstream libp11-0.4.17 (built from tag) fromdata_init=1 -> CLEAN
upstream libp11-0.4.18 (built from tag) fromdata_init=-2 -> POLLUTED
upstream libp11-0.4.21 (built from tag) fromdata_init=1 -> CLEAN
noble 0.4.12-1.1build2 (installed on resolute) fromdata_init=1 -> CLEAN
Fixed package expectation: fromdata_init=1 -> CLEAN.
Real-world verification: EAP-TLS via NetworkManager/wpa_supplicant with a
tpm2-pkcs11 token on 26.04; "wpa_cli status" should show EAP state=SUCCESS and
suppPortStatus=Authorized, including across repeated reconnects without
restarting wpa_supplicant (the breakage is process-lifetime state).
[ Where problems could occur ]
(Initial assessment from the reporter; to be refined by the uploader.)
- PR #675 does not apply cleanly to 0.4.18: about 120 upstream commits separate
0.4.18 and 0.4.21, and several hunks fail. The backport needs rework rather
than a cherry-pick. The related "Fix ENGINE pkey method lifetime" commit
(01c21ac, also in 0.4.21) may be needed alongside it.
- PR #675 adds a public API, PKCS11_CTX_set_pkey_callback(), which would be a
new exported symbol in libp11-3t64.
- An alternative narrower fix, reverting 60329e0, would drop RSA-PSS/RSA-OAEP
support for keys loaded via PKCS11_get_private_key(), i.e. return that path
to 0.4.17 behaviour. Not tested.
- Any regression would show up as failing RSA operations through the pkcs11
engine or libp11 API (signing tools, TLS clients using PKCS#11 keys).
[ Other Info ]
- Upstream issue: https://github.com/OpenSC/libp11/issues/672
- Upstream fix: https://github.com/OpenSC/libp11/pull/675
- Upstream release: libp11 0.4.21, https://github.com/OpenSC/libp11/releases
- Debian: libp11 0.4.21-1 in unstable and testing
- Related noble bug (same failure class, different cause): LP: #2163152
- Current workaround: install noble's libengine-pkcs11-openssl 0.4.12-1.1build2
on 26.04 and hold it, which loses RSA-PSS via the engine and means carrying a
24.04 build on 26.04.
[ Original report ]
Environment:
- Ubuntu 26.04.1 LTS, openssl 3.5.5, libengine-pkcs11-openssl 0.4.18-1
- wpasupplicant 2:2.11-0ubuntu5, NetworkManager, EAP-TLS
- TPM 2.0, tpm2-pkcs11 token via p11-kit
** Affects: libp11
Importance: Unknown
Status: Unknown
** Affects: libp11 (Ubuntu)
Importance: Undecided
Status: New
** Tags: regression-release resolute
** Bug watch added: github.com/OpenSC/libp11/issues #672
https://github.com/OpenSC/libp11/issues/672
** Also affects: libp11 via
https://github.com/OpenSC/libp11/issues/672
Importance: Unknown
Status: Unknown
** Tags added: resolute
** Tags added: regression-release
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2172829
Title:
EAP-TLS with TPM-backed PKCS#11 keys broken in 26.04: libp11 0.4.18
registers RSA EVP_PKEY_METHOD process-wide
To manage notifications about this bug go to:
https://bugs.launchpad.net/libp11/+bug/2172829/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
