The upgrade is now working fine in one of the applications which make TLS connections. I can see the engine functions being called when some action (sign/verify) which require the privatekey.
However, this engine is also used in a patched version of Racoon2. In one of the files (crypto_openssl.c) a function is called by the IKE daemon (iked) during setup with: : EVP_SignInit(&ctx, md); EVP_SignUpdate(&ctx, octets->v, octets->l); EVP_SignFinal(&ctx, (unsigned char*)sig->v, &siglen, pkey); : With the upgraded OpenSSL v1.0.2, the last function now fails with the error: 2017-08-28 15:44:14 [INTERNAL_ERR]: crypto_openssl.c:1238:eay_rsassa_pkcs1_v1_5_sign(): RSA_sign failed: 30972:error:0606B06E:digital envelope routines:EVP_SignFinal:wrong public key type:p_sign.c:123: Q: I assume it is looking for one of the missing parameters (p) in the RSA structure - correct? Q: If so, how did it work in v0.9.8? If I change the first command to: EVP_SignInit_ex(&ctx, md, engine); Then it segfaults in the "SignFinal" command :( The engine is dynamically loaded the same way in both my TLS connection application and in the Racoon2 application. Thanks for your time - any help is appreciated! Leon Brits System Engineer Mobile: +27 84 250 2855 [cid:image001.png@01D32016.91CF07F0] 76 Regency Drive Route 21 Corporate Park Irene 0157 Tel +27 12 678 9740 (ext. 9767) | Fax +27 12 345 2561 www.parsec.co.za<http://www.parsec.co.za> [cid:image002.png@01D32016.91CF07F0] From: openssl-dev [mailto:openssl-dev-boun...@openssl.org] On Behalf Of Leon Brits Sent: 28 August 2017 08:08 AM To: openssl-dev@openssl.org Subject: Re: [openssl-dev] Upgrading OpenSSL Thanks for the help. I've come to learn that my problem is the HSM. It removes the RSA values p,q and d from the EVP key before returning it. This is normal since it is protecting the key by keeping it in the HSM - duh. Anyway so, I cannot use it as a normal key. "Live and learn" So this bring me to the next question: Is there any changes I need to make in the OpenSSL Engine for my upgrade (0.9.8 -> 1.0.2) to be complete? Regards, Leon Brits System Engineer Mobile: +27 84 250 2855 [cid:image001.png@01D32016.91CF07F0] 76 Regency Drive Route 21 Corporate Park Irene 0157 Tel +27 12 678 9740 (ext. 9767) | Fax +27 12 345 2561 www.parsec.co.za<http://www.parsec.co.za> [cid:image002.png@01D32016.91CF07F0] From: openssl-dev [mailto:openssl-dev-boun...@openssl.org] On Behalf Of Leon Brits Sent: 23 August 2017 11:52 AM To: openssl-dev@openssl.org<mailto:openssl-dev@openssl.org> Subject: [openssl-dev] Upgrading OpenSSL Hi all, I am task to update two machines from v0.9.8z to v1.0.2 (since it is LTS). With the minimal changes, I've been able to get the application on the machines to compile with the newer version and generate RSA 4096 key pairs. The applications are able to successfully use their respective private keys and certificates to establish TLS connection between them. However, when I used the CLI to check a dumped privatekey i got the following output: % openssl rsa -check -in privkey.pem unable to load Private Key 1995859152:error:0D078079:asn1 encoding routines:ASN1_ITEM_EX_D2I:field missing:tasn_dec.c:489:Field=d, Type=RSA 1995859152:error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121: 1995859152:error:0606F091:digital envelope routines:EVP_PKCS82PKEY:private key decode error:evp_pkey.c:92: 1995859152:error:0907B00D:PEM routines:PEM_READ_BIO_PRIVATEKEY:ASN1 lib:pem_pkey.c:141: Any suggestions at what is wrong with the key? Note that an ID is stored in the RSA extended data since the private key may be stored in HSM. Thanks for your time LJB
-- openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev