Re: Load and unload of engines at runtime
Hi All, Could someone please help with this query? Thanks Mahendra On Thu, Apr 29, 2021 at 5:20 PM Mahendra SP wrote: > Hi All, > > We have crypto engines for offloading operations like RSA, digests and > ciphers, hmac etc. We are looking at a way to load and unload engines at > run time. This is needed as we need to use the engine when needed for > crypto operations. Else we plan to use openssl for the same. > > We tried, > -> unregister calls like ENGINE_unregister_XXX calls to force redirection > to openssl > -> Again, ENGINE_register_XXX to redirect to engine. > > However, the above methods are not helping. Please suggest a way to > achieve the above requirement. > > Thanks > Mahendra >
Load and unload of engines at runtime
Hi All, We have crypto engines for offloading operations like RSA, digests and ciphers, hmac etc. We are looking at a way to load and unload engines at run time. This is needed as we need to use the engine when needed for crypto operations. Else we plan to use openssl for the same. We tried, -> unregister calls like ENGINE_unregister_XXX calls to force redirection to openssl -> Again, ENGINE_register_XXX to redirect to engine. However, the above methods are not helping. Please suggest a way to achieve the above requirement. Thanks Mahendra
Re: Need inputs for engine cleanup
Hi All, Can you please let me know any inputs on the above query? Thanks Mahendra On Mon, Nov 30, 2020 at 9:19 PM Mahendra SP wrote: > Hi All, > > We are using the openssl 1.1.1 version and using the ENGINE implementation > for some crypto operation. Engine gets loaded dynamically and initialized > successfully and we are able to use the engine. > However, we plan to stop using this engine from the application side once > we are done with it. When we try to stop using the engine, our engine > references do not get removed. > We have tried this sequence: > ENGINE_free(); > ENGINE_finish(); > > However, ENGINE_remove() seems to remove the engine correctly and we see > that our engine does not get referred after this remove call. > > Can someone please provide a correct way of removing the engine so > that engine is no longer available for usage ? > > Thanks > Mahendra >
Need inputs for engine cleanup
Hi All, We are using the openssl 1.1.1 version and using the ENGINE implementation for some crypto operation. Engine gets loaded dynamically and initialized successfully and we are able to use the engine. However, we plan to stop using this engine from the application side once we are done with it. When we try to stop using the engine, our engine references do not get removed. We have tried this sequence: ENGINE_free(); ENGINE_finish(); However, ENGINE_remove() seems to remove the engine correctly and we see that our engine does not get referred after this remove call. Can someone please provide a correct way of removing the engine so that engine is no longer available for usage ? Thanks Mahendra
Question related to default RAND usage and update with engine RAND
Hi All,
We are planning to use our own RAND implementation using an engine. What we
observe is, during Openssl init, default RAND gets initialized to openssl
RAND.
Then later we initialize our engine RAND. Even though we make our RAND as
default, we see that still openssl uses the initial default RAND.
Here is what could be happening. In the function RAND_get_rand_method,
default_RAND_meth gets initialized to openssl RAND.
As there is a NULL check for default_RAND_meth , default_RAND_meth never
gets updated as it is not NULL.
Even if engine RAND is registered and available for use, default_RAND_meth
never gets updated.
Given the code snippet below.
const RAND_METHOD *RAND_get_rand_method(void)
{
const RAND_METHOD *tmp_meth = NULL;
if (!RUN_ONCE(_init, do_rand_init))
return NULL;
CRYPTO_THREAD_write_lock(rand_meth_lock);
if (default_RAND_meth == NULL) {
#ifndef OPENSSL_NO_ENGINE
ENGINE *e;
/* If we have an engine that can do RAND, use it. */
if ((e = ENGINE_get_default_RAND()) != NULL
&& (tmp_meth = ENGINE_get_RAND(e)) != NULL) {
funct_ref = e;
default_RAND_meth = tmp_meth;
} else {
ENGINE_finish(e);
default_RAND_meth = _meth;
}
#else
default_RAND_meth = _meth;
#endif
}
tmp_meth = default_RAND_meth;
CRYPTO_THREAD_unlock(rand_meth_lock);
return tmp_meth;
}
Should we remove the NULL check for default_RAND_meth to fix this issue ?
Or is there any other way?
Thanks
Mahendra
Re: Decrypt error when using openssl 1.1.1b during SSL handshake
Hi Matt, Error is reported from this: FILE:../openssl-1.1.1b/ssl/statem/statem_srvr.c, FUNCTION:415, LINE:3055, reason=147, alert=51 We see that hardware is returning 48 bytes. Even if the decrypted premaster data is correct, openssl is expecting more than 48 bytes in return. This check fails as decrypt_len is 48. decrypt_len < 11 + SSL_MAX_MASTER_KEY_LENGTH We compared the data returned when software is used. Decrypt_len is 256 bytes and the last 48 bytes are actual premaster secret. Also, openssl checks for if the first byte is 0 and second byte is 2. We are trying to rectify this issue in hardware and return the correct data. Please suggest if you have any comments for the above info. Thanks Mahendra On Fri, Oct 30, 2020 at 7:50 PM Matt Caswell wrote: > > > On 30/10/2020 11:22, Mahendra SP wrote: > > Hi Matt, > > > > Thank you for the inputs. > > Yes, we had encountered the padding issue initially. But we added > > support for RSA_NO_PADDING in our hardware. That's why we are able to > > successfully decrypt the premaster secret in the hardware. > > Hence the issue does not seem to be related to padding. We have > > confirmed this by comparing the premaster secret on both client and > > server and they are the same. > > Ok, good. > > > > > We suspect in this case, verification of "encrypted handshake message" > > failure is happening. > > It's possible. It would be helpful if you can get more information from > the error stack on the server, e.g. by using ERR_print_errors_fp() or > something similar. I'm particularly interested in identifying the source > file and line number where the decrypt_error is coming from. Printing > the error stack should give us that information. There are a number of > places that a "decrypt error" can occur and it would be helpful to > identify which one is the cause of the problem. > > > > We understand constant_time_xx APIs get used for CBC padding validation. > > Will this have any dependency on the compiler optimization or asm > > flags? > > CBC padding validation is fairly independent of anything to do with RSA, > so I think its unlikely to be the culprit here. Of course sometimes > compiler optimization/asm issues do occur so it can't be ruled out > entirely - but it's not where I would start looking. > > > Will this issue be seen if hardware takes more time for the > > operation? > > > > No. Constant time here just means that we implement the code without any > branching based on secret data (e.g. no "if" statements/while loops etc > based on secret dependent data). It has very little to do with how long > something actually takes to process. > > > > Here is the snippet of the wireshark where our device acting as server > > closes the connection with decryption failure. > > Thanks. To narrow it down further I need to figure out which line of > code the decrypt_error is coming from as described above. > > Matt > > > > > If you need any further info, please let us know. > > image.png > > Thanks > > Mahendra > > > > Please suggest. > > > > > > > > On Fri, Oct 30, 2020 at 3:32 PM Matt Caswell > <mailto:m...@openssl.org>> wrote: > > > > > > > > On 30/10/2020 09:18, Mahendra SP wrote: > > > Hi All. > > > > > > We have upgraded openssl version to 1.1.1b > > > > > > With this, we are seeing decryption error during SSL handshake for > the > > > below explained scenario. Our device acts as an SSL server. > > > > > > We have external hardware to offload RSA private key operations > using > > > the engine. > > > Decryption of pre-master secret is done using hardware and is > > > successful. We compared the pre-master secret on both server and > > client > > > and they match. > > > However, we see that SSL handshake fails with "decrypt error (51)" > > with > > > an alert number 21. Verifying the encrypted finish message on the > > server > > > side fails. > > > > > > This issue does not happen with software performing RSA private key > > > operations. > > > > > > Can someone help with the reason for decryption failure? Below is > the > > > compiler and processor details. It is 64 bit. > > > arm-linux-gnueabihf-gcc -march=armv7ve -mthumb -mfpu=neon > > -mfloat-abi=hard > > > > Potentially this is related to the use of PSS padding in libssl > w
Re: Decrypt error when using openssl 1.1.1b during SSL handshake
Hi Matt, Thank you for the inputs. Yes, we had encountered the padding issue initially. But we added support for RSA_NO_PADDING in our hardware. That's why we are able to successfully decrypt the premaster secret in the hardware. Hence the issue does not seem to be related to padding. We have confirmed this by comparing the premaster secret on both client and server and they are the same. We suspect in this case, verification of "encrypted handshake message" failure is happening. We understand constant_time_xx APIs get used for CBC padding validation. Will this have any dependency on the compiler optimization or asm flags? Will this issue be seen if hardware takes more time for the operation? Here is the snippet of the wireshark where our device acting as server closes the connection with decryption failure. If you need any further info, please let us know. [image: image.png] Thanks Mahendra Please suggest. On Fri, Oct 30, 2020 at 3:32 PM Matt Caswell wrote: > > > On 30/10/2020 09:18, Mahendra SP wrote: > > Hi All. > > > > We have upgraded openssl version to 1.1.1b > > > > With this, we are seeing decryption error during SSL handshake for the > > below explained scenario. Our device acts as an SSL server. > > > > We have external hardware to offload RSA private key operations using > > the engine. > > Decryption of pre-master secret is done using hardware and is > > successful. We compared the pre-master secret on both server and client > > and they match. > > However, we see that SSL handshake fails with "decrypt error (51)" with > > an alert number 21. Verifying the encrypted finish message on the server > > side fails. > > > > This issue does not happen with software performing RSA private key > > operations. > > > > Can someone help with the reason for decryption failure? Below is the > > compiler and processor details. It is 64 bit. > > arm-linux-gnueabihf-gcc -march=armv7ve -mthumb -mfpu=neon > -mfloat-abi=hard > > Potentially this is related to the use of PSS padding in libssl which is > mandated in TLSv1.3. The TLSv1.3 spec also requires its use even in > TLSv1.2. > > The PSS padding is implemented within the EVP layer. Ultimately EVP > calls the function RSA_private_encrypt() with padding set to > RSA_NO_PADDING. > > Assuming your engine is implemented via a custom RSA_METHOD does it > support RSA_private_encrypt(() with RSA_NO_PADDING? If not this is > likely to be the problem. > > More discussion of this is here: > > https://github.com/openssl/openssl/issues/7968 > > Also related is the recent discussion on this list about the CAPI engine > and this issue: > > https://github.com/openssl/openssl/issues/8872 > > Matt >
Decrypt error when using openssl 1.1.1b during SSL handshake
Hi All. We have upgraded openssl version to 1.1.1b With this, we are seeing decryption error during SSL handshake for the below explained scenario. Our device acts as an SSL server. We have external hardware to offload RSA private key operations using the engine. Decryption of pre-master secret is done using hardware and is successful. We compared the pre-master secret on both server and client and they match. However, we see that SSL handshake fails with "decrypt error (51)" with an alert number 21. Verifying the encrypted finish message on the server side fails. This issue does not happen with software performing RSA private key operations. Can someone help with the reason for decryption failure? Below is the compiler and processor details. It is 64 bit. arm-linux-gnueabihf-gcc -march=armv7ve -mthumb -mfpu=neon -mfloat-abi=hard Thanks Mahendra
Re: ENGINE_load_private_key does not get invoked
Hi Dmitry Belyavsky,
Thank you..To give more info, I am looking at something similar the engine
in e_4758cca.c in engines folder where ENGINE_set_load_privkey_function is
called.
My understanding was, once the callback is set, it automatically gets
called during RSA operations. As demostrated in e_4758cca.c, ex data can be
added and during private key operations, same can be access to decide the
operation. Is this not correct ?
>From your above inputs, looks like ENGINE_set_load_privkey_function needs
to be called to invoke the callback.
e_4758cca.c shows that in callback, RSA can be modified to add the ex data
so that same can be accessed during RSA private key operations.
To confirm, once the callback is registered using
ENGINE_set_load_privkey_function, callback gets called by openssl before
every private key operation. Is this correct ?
Thanks
Mahendra
On Mon, Apr 20, 2020 at 10:03 PM Dmitry Belyavsky wrote:
> Dear Mahendra,
>
> On Mon, Apr 20, 2020 at 7:27 PM Mahendra SP wrote:
>
>> Hi Dmitry Belyavsky,
>>
>> Thank you for the inputs. If I understand correctly, the
>> reference indicates loading the private key to engine instance.
>>
>> My requirement is to call the ENGINE_set_load_privkey_function so that
>> the callback gets called before each RSA private key operation.
>> Reason being, application can use either key with TPM or keys in
>> software. So I need to load the correct key before each operation gets
>> invoked.I was looking at
>> ENGINE_set_load_privkey_function to set the callback.Is this
>> understanding correct ?
>>
>
> You specify the engine-specific function via
> ENGINE_set_load_privkey_function.
>
> After that, you can load your key using ENGINE_load_private_key -
> if you pass the reference to your engine to it, the function set via
> ENGINE_set_load_privkey_function will be used.
>
>
>> Please suggest.
>>
>> Thanks
>> Mahendra
>>
>>
>>
>> On Mon, Apr 20, 2020 at 8:43 PM Dmitry Belyavsky
>> wrote:
>>
>>> Dear Mahendra,
>>>
>>> Take a look at the load_key function in the apps/apps.c as a referent
>>> example.
>>>
>>> On Mon, Apr 20, 2020 at 2:34 PM Mahendra SP
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> While writing the engine implementation for private key encryption and
>>>> decryption, I need to get the private key from external source.
>>>> ENGINE_set_load_privkey_function(e, load_tpm_private_key)
>>>> Above API is success.
>>>>
>>>> RSA structure is as below.
>>>> RSA_METHOD my_rsa_struct =
>>>> {
>>>>"MY RSA method",
>>>>NULL,
>>>>NULL,
>>>>MyRSAPrivEnc,
>>>>MyRSAPrivDec,
>>>>NULL,
>>>>NULL,
>>>>NULL,
>>>>NULL,
>>>>0,
>>>>NULL,
>>>>NULL,
>>>>NULL,
>>>>NULL
>>>> };
>>>>
>>>> However, the routine load_tpm_private_key does not get invoked during
>>>> private key encryption and decryption.
>>>> Private key encryption and decryption gets routed to the above
>>>> overloaded routines namely " MyRSAPrivEnc " and " MyRSAPrivDec". But
>>>> load_tpm_private_key never gets invoked.
>>>>
>>>> Is there a flag to force the same ? I have tried it using openssl
>>>> 1.0.2 version. All other engine calls work as expected, but not this one.
>>>>
>>>> I have also tried to debug in ENGINE_load_private_key in eng_pkey.c
>>>> file. I dont see this function getting invoked. In this function, the
>>>> private key callback set above gets invoked.
>>>>
>>>> Please suggest if I am missing something here.
>>>>
>>>> Thanks
>>>> Mahendra
>>>>
>>>
>>>
>>> --
>>> SY, Dmitry Belyavsky
>>>
>>
>
> --
> SY, Dmitry Belyavsky
>
Re: ENGINE_load_private_key does not get invoked
Hi Dmitry Belyavsky,
Thank you for the inputs. If I understand correctly, the
reference indicates loading the private key to engine instance.
My requirement is to call the ENGINE_set_load_privkey_function so that the
callback gets called before each RSA private key operation.
Reason being, application can use either key with TPM or keys in software.
So I need to load the correct key before each operation gets invoked.I was
looking at
ENGINE_set_load_privkey_function to set the callback.Is this understanding
correct ?
Please suggest.
Thanks
Mahendra
On Mon, Apr 20, 2020 at 8:43 PM Dmitry Belyavsky wrote:
> Dear Mahendra,
>
> Take a look at the load_key function in the apps/apps.c as a referent
> example.
>
> On Mon, Apr 20, 2020 at 2:34 PM Mahendra SP wrote:
>
>> Hi All,
>>
>> While writing the engine implementation for private key encryption and
>> decryption, I need to get the private key from external source.
>> ENGINE_set_load_privkey_function(e, load_tpm_private_key)
>> Above API is success.
>>
>> RSA structure is as below.
>> RSA_METHOD my_rsa_struct =
>> {
>>"MY RSA method",
>>NULL,
>>NULL,
>>MyRSAPrivEnc,
>>MyRSAPrivDec,
>>NULL,
>>NULL,
>>NULL,
>>NULL,
>>0,
>>NULL,
>>NULL,
>>NULL,
>>NULL
>> };
>>
>> However, the routine load_tpm_private_key does not get invoked during
>> private key encryption and decryption.
>> Private key encryption and decryption gets routed to the above overloaded
>> routines namely " MyRSAPrivEnc " and " MyRSAPrivDec". But
>> load_tpm_private_key never gets invoked.
>>
>> Is there a flag to force the same ? I have tried it using openssl 1.0.2
>> version. All other engine calls work as expected, but not this one.
>>
>> I have also tried to debug in ENGINE_load_private_key in eng_pkey.c file.
>> I dont see this function getting invoked. In this function, the private key
>> callback set above gets invoked.
>>
>> Please suggest if I am missing something here.
>>
>> Thanks
>> Mahendra
>>
>
>
> --
> SY, Dmitry Belyavsky
>
ENGINE_load_private_key does not get invoked
Hi All,
While writing the engine implementation for private key encryption and
decryption, I need to get the private key from external source.
ENGINE_set_load_privkey_function(e, load_tpm_private_key)
Above API is success.
RSA structure is as below.
RSA_METHOD my_rsa_struct =
{
"MY RSA method",
NULL,
NULL,
MyRSAPrivEnc,
MyRSAPrivDec,
NULL,
NULL,
NULL,
NULL,
0,
NULL,
NULL,
NULL,
NULL
};
However, the routine load_tpm_private_key does not get invoked during
private key encryption and decryption.
Private key encryption and decryption gets routed to the above overloaded
routines namely " MyRSAPrivEnc " and " MyRSAPrivDec". But
load_tpm_private_key never gets invoked.
Is there a flag to force the same ? I have tried it using openssl 1.0.2
version. All other engine calls work as expected, but not this one.
I have also tried to debug in ENGINE_load_private_key in eng_pkey.c file. I
dont see this function getting invoked. In this function, the private key
callback set above gets invoked.
Please suggest if I am missing something here.
Thanks
Mahendra
Regarding engine implementation for RSA and AES
Hi, We are looking at implementing an engine for RSA and AES. For AES, looking at the documentation, understand that engine will get invoked only if the application or SSL library invokes EVP based calls rather than native AES calls. Is this correct ? For RSA, is it mandatory to use EVP for using engine ? Openssl version dependency: Is it a must to use latest openssl version for engine ? Reason for this question is, looking at 1.1.X code, all the calls are modified to EVP in SSL library when compared to previous versions. As understanding is, engine works with EVP, wanted to know if we need to use latest version. Thanks Mahendra
Re: Query regarding SSL_ERROR_SSL during SSL handshake
Hi Matt, Thank you for the inputs. I have one more query. Is it appropriate to check for the errno in this case and take action based on the errno values ? Thanks Mahendra On Wed, Feb 19, 2020 at 3:09 PM Matt Caswell wrote: > > > On 19/02/2020 05:16, Mahendra SP wrote: > > Hi All, > > > > We are using Openssl version 1.0.2h. When we call SSL_do_handshake, > > sometimes we notice that handshake fails with error SSL_ERROR_SSL. > > As per the documentation for this error, it is non recoverable and fatal > > error. Documentation also mentions to check the error queue for further > > details. Does it mean, calling SSL_get_error after SSL_ERROR_SSL will > > give exact reason for this failure? > > OpenSSL has its own error stack. SSL_ERROR_SSL means that you should > look at that error stack for further details about what caused the > problem. For example you can use ERR_print_errors_fp() to print all the > error descriptions to stdout/stderr: > > https://www.openssl.org/docs/man1.1.1/man3/ERR_print_errors_fp.html > > You can get more fine grained control of the error stack using the > various ERR_* functions available. See: > > https://www.openssl.org/docs/man1.1.1/man3/ > > Matt >
Query regarding SSL_ERROR_SSL during SSL handshake
Hi All, We are using Openssl version 1.0.2h. When we call SSL_do_handshake, sometimes we notice that handshake fails with error SSL_ERROR_SSL. As per the documentation for this error, it is non recoverable and fatal error. Documentation also mentions to check the error queue for further details. Does it mean, calling SSL_get_error after SSL_ERROR_SSL will give exact reason for this failure? Could you please let me know ways to identify the cause of this error? Thanks Mahendra
Issue in CRYPTO_cbc128_encrypt due to unaligned address
Hi All, I am seeing an issue with the routine CRYPTO_cbc128_encrypt() in version 1.0.2h where, , if I have not declared STRICT_ALIGNMENT, application crashes intermittently. There was a similar query here but for CRYPTO_ctr128_encrypt. https://mta.openssl.org/pipermail/openssl-dev/2016-January/003966.html I have dumped the addresses for both ivec and iv. Even though the addresses are unaligned most of the times, crash is seen only occasioanally. I don't want to declare STRICT_ALIGNMENT for performance reasons as I need to have the size_t based operations. Is there a work around similar to one proposed in the above thread or any other suggestions on how to handle this ? Thanks Mahendra
