Reducing the footprint of a simple application

[Reinier Torenbeek]

Hi,

I have a simple application that uses OpenSSL 3.0.0 for AES-GCM encryption
and decryption only. Looking at the size of the binary on disc, I see it's
a few KBs when linking dynamically with libcrypto, and  4.8 MB when linking
statically. Although I know the large footprint of OpenSSL is considered "a
fact of life", this seems excessive. From experience with other crypto
implementations, I know that the *actual* crypto functionality that I am
using can fit in 10s of KBs and I would like to understand the reasons for
OpenSSL's size better. I am on Linux, 64 bits, using gcc 9.3.0.

Some analysis of the binary reveals (not surprisingly) that it contains
many OpenSSL symbols that have nothing to do with the functionality that I
am using. Those seem to get pulled in because objects get linked in as a
whole and apparently the layout of the object contents are such that the
symbols needed for my functionality are spread out over many different
objects.

It was my hope that I could mitigate this by compiling OpenSSL and my
application with the flags -ffunction-sections, -fdata-sections, -Os and
-flto and using --gc-sections and -flto when linking. (See 3.10 Options
That Control Optimization

of
GCC's documentation).  This did reduce the binary size by 2 MB, leaving me
with almost 3 MB. Almost 90% of that was in the text section and a bit over
10% in the data section. I do not have sufficient experience with these
options to assess how well the optimizations worked in this case, I think
the resulting binary is still pretty large.

I have not tried disabling any of the features when building OpenSSL. I
suspect that may help a little bit because it may result in a decrease in
size of (some) objects, but I have seen people reporting
disappointing results of that on the web. Also, it does not seem to be a
workable approach in general to have to figure out which build options to
use and to have to rebuild OpenSSL for every type of application that I
create.

Did any people here try similar things, with better results? Does anybody
have any other suggestions as to what I could try? And what is the
explanation (or justification) for this excessive footprint?

Thanks,
Reinier

Re: Several _METHOD structs not marked as deprecated in 3.0.0-beta2

[Reinier Torenbeek]

Hi,
This seems to be an oversight so I created issue #16272: Several legacy
structs are not marked as deprecated
<https://github.com/openssl/openssl/issues/16272>.
Reinier

On Sun, Aug 8, 2021 at 10:52 PM Reinier Torenbeek <
reinier.torenb...@gmail.com> wrote:

> Hi,
>
> While checking out the 3.0.0-beta2 release, I noticed that several of the
> _METHOD structs are not marked as deprecated. With the introduction of the
> provider mechanism, they don't seem relevant anymore. Is there a reason for
> this, or is it just an omission?
>
> For example, see DH_METHOD here:
> https://github.com/openssl/openssl/blob/9f551541e84eead1d42604b7d5e61885e8e34be0/include/openssl/types.h#L134-L135
> Others I have seen are RAND_METHOD and EVP_PKEY_METHOD.
>
> Thanks,
> Reinier
>

Several _METHOD structs not marked as deprecated in 3.0.0-beta2

[Reinier Torenbeek]

Hi,

While checking out the 3.0.0-beta2 release, I noticed that several of the
_METHOD structs are not marked as deprecated. With the introduction of the
provider mechanism, they don't seem relevant anymore. Is there a reason for
this, or is it just an omission?

For example, see DH_METHOD here:
https://github.com/openssl/openssl/blob/9f551541e84eead1d42604b7d5e61885e8e34be0/include/openssl/types.h#L134-L135
Others I have seen are RAND_METHOD and EVP_PKEY_METHOD.

Thanks,
Reinier

Re: CNG engine on GitHub

[Reinier Torenbeek]

Hello Selva and Matt,

Thanks for the pointers. Following the suggested approach, I have added (in
a branch of a fork) initial support of RSA-PSS for the BCrypt engine and
the few first tests look promising. Next, I will do the same thing for
NCrypt. After that I will probably add support for OAEP as well.

Best regards,
Reinier

On Fri, Jul 2, 2021 at 1:35 PM Selva Nair  wrote:

> Hi
>
>>
>>>
>>> This is great, but limiting RSA signature to  RSA-PKCS#1 v 1.5 is a
>>> major limitation. It doesn't have to be that way as the OpenSSL engine
>>> interface does allow using EVP_PKEY_METHOD callbacks instead of
>>> rsa_priv_dec etc.
>>>
>>
>> Yes I agree the lack of support for RSA-PSS is significant. There is a
>> discussion (which includes you, I see ) around the root cause of that here:
>> https://github.com/openssl/openssl/issues/7341 , among other places.
>>
>
> That discussion is valid only if you insist on using "legacy" rsa_sign or
> other rsa_priv_dec  which have no mechanism for providing context info like
> padding and hash type.
>
>
>> It is not clear to me what you mean with "the OpenSSL engine interface
>> does allow using EVP_PKEY_METHOD callbacks instead of rsa_priv_dec etc.".
>> Can you elaborate (here or on the GitHub issue)?
>>
>
> To add to what Matt wrote:
>
> As an example, see my PR for pkcs11-helper:
> https://github.com/OpenSC/pkcs11-helper/pull/31  This uses a dummy engine
> and passes the singing operation to the pkcs11 device, but the idea is the
> same.
>
> Selva
>
>>

Re: CNG engine on GitHub

[Reinier Torenbeek]

Hi Selva,

On Fri, Jul 2, 2021 at 10:49 AM Selva Nair  wrote:

> Hi,
>
> On Thu, Jul 1, 2021 at 1:49 PM Reinier Torenbeek <
> reinier.torenb...@gmail.com> wrote:
>
>> Hi,
>>
>> For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1, you
>> may want to check out this new OpenSSL CNG Engine project on GitHub:
>> https://github.com/rticommunity/openssl-cng-engine . The associated
>> User's Manual is on ReadTheDocs:
>> https://openssl-cng-engine.readthedocs.io/en/latest/index.html .
>>
>> The project implements the majority of the EVP interface, to leverage the
>> BCrypt crypto implementations, as well as a subset of the STORE interface,
>> for integration with the Windows Certificate and Keystore(s), via the
>> NCrypt and Cert APIs. It has been tested with 1.1.1k on Windows 10, with
>> Visual Studio 2017 and 2019. It is released under the Apache-2.0 license.
>>
>> Any feedback is welcome, please send it to me or open an issue on GitHub.
>>
>
> This is great, but limiting RSA signature to  RSA-PKCS#1 v 1.5 is a major
> limitation. It doesn't have to be that way as the OpenSSL engine interface
> does allow using EVP_PKEY_METHOD callbacks instead of rsa_priv_dec etc.
>

Yes I agree the lack of support for RSA-PSS is significant. There is a
discussion (which includes you, I see ) around the root cause of that here:
https://github.com/openssl/openssl/issues/7341 , among other places.

It is not clear to me what you mean with "the OpenSSL engine interface does
allow using EVP_PKEY_METHOD callbacks instead of rsa_priv_dec etc.". Can
you elaborate (here or on the GitHub issue)?

Thanks,
Reinier


> Selva
>
>>

Re: CNG engine on GitHub

[Reinier Torenbeek]

Hi Richard,

Glad you like it. The cert: scheme is a little inconvenient and I do not
know how extensively it is used in practice. But it seemed appropriate to
leverage it since it was around already and seemed to fit the bill.

Microsoft's documentation is not too extensive, but for anybody interested,
here is a starting point:
https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.security/about/about_certificate_provider?view=powershell-7.1

Best regards,
Reinier

On Fri, Jul 2, 2021 at 9:03 AM Richard Levitte  wrote:

> This is cool!
>
> I had some kind of skeleton of a start to make something similar, but
> time was never on my side.  I'm really glad to see this got picked up!
>
> This also answered a question I never got the answer for, what scheme
> to use for the STORE.  I know next to nothing about PowerShell, so
> hadn't discovered the 'cert:' "scheme".  That answers quite a lot :-)
>
> Time for me to throw away my skeleton then ;-)
>
> Cheers,
> Richard
>
> On Thu, 01 Jul 2021 19:49:00 +0200,
> Reinier Torenbeek wrote:
> >
> > Hi,
> >
> > For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1, you
> may want to check out this
> > new OpenSSL CNG Engine project on GitHub:
> https://github.com/rticommunity/openssl-cng-engine . The
> > associated User's Manual is on ReadTheDocs:
> > https://openssl-cng-engine.readthedocs.io/en/latest/index.html .
> >
> > The project implements the majority of the EVP interface, to leverage
> the BCrypt crypto
> > implementations, as well as a subset of the STORE interface, for
> integration with the
> > Windows Certificate and Keystore(s), via the NCrypt and Cert APIs. It
> has been tested with 1.1.1k
> > on Windows 10, with Visual Studio 2017 and 2019. It is released under
> the Apache-2.0 license.
> >
> > Any feedback is welcome, please send it to me or open an issue on GitHub.
> >
> > Best regards,
> > Reinier
> >
> >
> --
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/
>

Re: OpenSSL CNG engine on GitHub

[Reinier Torenbeek]

Hello David,

Thanks for checking this out and your positive feedback. I was not able to
find any substantial solution for this either. I do wonder why that is?
Possibly, Windows users are not as interested in a cross platform solution
like OpenSSL provides and they are fine with using the Windows APIs
directly -- that is just speculation though.

Best regards,
Reinier

On Fri, Jul 2, 2021 at 6:56 AM David von Oheimb  wrote:

> Hello Reinier,
>
> around five years back I was looking for such an implementation as an
> alternative to the rather limited CAPI engine, mostly because the C(rypto
> )API does not support ECC.
> The only thing I found at that time was
> https://mta.openssl.org/pipermail/openssl-dev/2016-June/007362.html and I
> do not know how it evolved since them.
> So I am very pleased to see that meanwhile there is a way of using core
> features of Windows CAPI Next Generation (CNG) from OpenSSL.
>
> Many thanks to RTI for providing this as open-source development under the
> Apache license.
> I currently do not have the time for a closer look or even trying it out,
> but this looks very good and well documented.
> In particular,
> https://openssl-cng-engine.readthedocs.io/en/latest/using/openssl_commands.html
> gives a nice example how to use the Windows cert & key store.
> Porting this to the new OpenSSL crypto provider interface will likely lift
> the limitation regarding RSA-PSS support, which lacks just due to the
> engine interface.
>
> Cheers,
>
> David
>
>
> On 01.07.21 19:49, Reinier Torenbeek wrote:
>
> Hi,
>
> For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1, you
> may want to check out this new OpenSSL CNG Engine project on GitHub:
> https://github.com/rticommunity/openssl-cng-engine . The associated
> User's Manual is on ReadTheDocs:
> https://openssl-cng-engine.readthedocs.io/en/latest/index.html .
>
> The project implements the majority of the EVP interface, to leverage the
> BCrypt crypto implementations, as well as a subset of the STORE interface,
> for integration with the Windows Certificate and Keystore(s), via the
> NCrypt and Cert APIs. It has been tested with 1.1.1k on Windows 10, with
> Visual Studio 2017 and 2019. It is released under the Apache-2.0 license.
>
> Any feedback is welcome, please send it to me or open an issue on GitHub.
>
> Best regards,
> Reinier
>
>

Re: CNG engine on GitHub

[Reinier Torenbeek]

Thanks Matt.
>From your response, it seems that this would be a good moment to start
looking into the provider interface. I will check it out (and may get back
with questions after that...)
Reinier

On Fri, Jul 2, 2021 at 4:21 AM Matt Caswell  wrote:

>
>
> On 02/07/2021 04:25, Reinier Torenbeek wrote:
> > Hi Matt,
> >
> > I am aware of the deprecation of the engine interface with 3.0 but have
> not looked into the details of support providers yet. I  expect converting
> an engine to a support provider could be done with quite a bit of code
> reuse, correct? Would you say the interface and design of support providers
> is stable at this point?
>
> The engine and provider interfaces are quite different - but since the
> underlying operations are the same I imagine there will be quite a bit
> of reuse.
>
> Yes, we consider the provider interface to be stable now.
>
> Matt
>
>
> >
> > Thanks,
> > Reinier
> >
> >> On Jul 1, 2021, at 4:41 PM, Matt Caswell  wrote:
> >>
> >> Nice! Are there any thoughts to support providers? The engine
> interface is deprecated in 3.0.
> >>
> >> Matt
> >>
> >>
> >>> On 01/07/2021 18:49, Reinier Torenbeek wrote:
> >>> Hi,
> >>> For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1,
> you may want to check out this new OpenSSL CNG Engine project on GitHub:
> https://github.com/rticommunity/openssl-cng-engine <
> https://github.com/rticommunity/openssl-cng-engine> . The associated
> User's Manual is on ReadTheDocs:
> https://openssl-cng-engine.readthedocs.io/en/latest/index.html <
> https://openssl-cng-engine.readthedocs.io/en/latest/index.html> .
> >>> The project implements the majority of the EVP interface, to leverage
> the BCrypt crypto implementations, as well as a subset of the STORE
> interface, for integration with the Windows Certificate and Keystore(s),
> via the NCrypt and Cert APIs. It has been tested with 1.1.1k on Windows 10,
> with Visual Studio 2017 and 2019. It is released under the Apache-2.0
> license.
> >>> Any feedback is welcome, please send it to me or open an issue on
> GitHub.
> >>> Best regards,
> >>> Reinier
> >
>

Re: CNG engine on GitHub

[Reinier Torenbeek]

Hi Matt,

I am aware of the deprecation of the engine interface with 3.0 but have not 
looked into the details of support providers yet. I  expect converting an 
engine to a support provider could be done with quite a bit of code reuse, 
correct? Would you say the interface and design of support providers is stable 
at this point?

Thanks,
Reinier

> On Jul 1, 2021, at 4:41 PM, Matt Caswell  wrote:
> 
> Nice! Are there any thoughts to support providers? The engine interface is 
> deprecated in 3.0.
> 
> Matt
> 
> 
>> On 01/07/2021 18:49, Reinier Torenbeek wrote:
>> Hi,
>> For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1, you may 
>> want to check out this new OpenSSL CNG Engine project on GitHub: 
>> https://github.com/rticommunity/openssl-cng-engine 
>> <https://github.com/rticommunity/openssl-cng-engine> . The associated User's 
>> Manual is on ReadTheDocs: 
>> https://openssl-cng-engine.readthedocs.io/en/latest/index.html 
>> <https://openssl-cng-engine.readthedocs.io/en/latest/index.html> .
>> The project implements the majority of the EVP interface, to leverage the 
>> BCrypt crypto implementations, as well as a subset of the STORE interface, 
>> for integration with the Windows Certificate and Keystore(s), via the NCrypt 
>> and Cert APIs. It has been tested with 1.1.1k on Windows 10, with Visual 
>> Studio 2017 and 2019. It is released under the Apache-2.0 license.
>> Any feedback is welcome, please send it to me or open an issue on GitHub.
>> Best regards,
>> Reinier

CNG engine on GitHub

[Reinier Torenbeek]

Hi,

For anyone interested in leveraging Windows CNG with OpenSSL 1.1.1, you may
want to check out this new OpenSSL CNG Engine project on GitHub:
https://github.com/rticommunity/openssl-cng-engine . The associated User's
Manual is on ReadTheDocs:
https://openssl-cng-engine.readthedocs.io/en/latest/index.html .

The project implements the majority of the EVP interface, to leverage the
BCrypt crypto implementations, as well as a subset of the STORE interface,
for integration with the Windows Certificate and Keystore(s), via the
NCrypt and Cert APIs. It has been tested with 1.1.1k on Windows 10, with
Visual Studio 2017 and 2019. It is released under the Apache-2.0 license.

Any feedback is welcome, please send it to me or open an issue on GitHub.

Best regards,
Reinier

Re: [openssl-users] Freeing of const char * fields in ASN1_OBJECT

[Reinier Torenbeek]

Hi Michael,

Many thanks for the extensive explanation, I understand the rationale now.

> Now, a *good* public C library will implement proper ADTs.

Indeed I am used to using ADT mechanisms so I did not run into this
issue anywhere else before. It will be a lot of work to make all types
abstracted in OpenSSL... However, as a compromise, I wonder how much
work it would be to at least provide encapsulating allocator/initializer
and free functions for all datatypes. In that case, the const qualifier
for some fields in the struct would make complete sense and the fact
that the const field is allocated/freed by the library is then no longer
anybody else's business.

My problem was indeed resolved by using OBJ_create(), so from now on I
will be looking for creation functions rather than assuming that I am
supposed/allowed to create my own structs directly.

Reinier

PS: apparently there are more reasons to cast away const-ness; I ran
into the following in objects/obj_lib.c:

 65 ASN1_OBJECT *OBJ_dup(const ASN1_OBJECT *o)
 66 {
 67 ASN1_OBJECT *r;
 68 int i;
 69 char *ln=NULL,*sn=NULL;
 70 unsigned char *data=NULL;
 71
 72 if (o == NULL) return(NULL);
 73 if (!(o->flags & ASN1_OBJECT_FLAG_DYNAMIC))
 74 return((ASN1_OBJECT *)o); /* XXX: ugh! Why? What kind of
 75  duplication is this??? */

On 10/28/15 11:02 PM, Michael Wojcik wrote:
>> From: openssl-users [mailto:openssl-users-boun...@openssl.org] On Behalf
>> Of Reinier Torenbeek
>> Sent: Wednesday, October 28, 2015 16:55
>>
>> In asn1/asn1.h, the fields sn and ln in ASN1_OBJECT are defined as const
>> char *:
>>
>> 211 typedef struct asn1_object_st
>> 212 {
>> 213 const char *sn,*ln;
>> 214 int nid;
>> 215 int length;
>> 216 const unsigned char *data;  /* data remains const after
>> init */
>> 217 int flags;  /* Should we free this one */
>> 218 } ASN1_OBJECT;
>>
>> but in asn1/a_object.c, the get casted to void * and freed:
>> ...
>>
>> Given that a lot of the code is supposed to be self-describing (due to
>> lack of documentation), it is somewhat disturbing that I can not rely on
>> the const qualifiers to be honored. Any thoughts on this?
> It's a well-known problem with the semantics of const in C.
>
> The parameter to free() cannot have the const qualifier, because the 
> semantics of free say that the object referred to by the pointer reaches the 
> end of its lifetime during the call to free(), and attempting to use it after 
> that results in Undefined Behavior.
>
> However, it is perfectly reasonable to want to allocate storage, initialize 
> (and even alter) its contents, but have that object treated only as an rvalue 
> elsewhere. This is a very typical use case for a library or other code that 
> presents an ADT (abstract data type) to a caller.
>
> The mechanism for informing a caller that an object should not be modified is 
> the const qualifier.
>
> There are only two ways around this problem. One is for the ADT 
> implementation to keep both a const and a non-const pointer to the same area, 
> publish the const one, and use the non-const one for freeing. That's a rather 
> absurd duplication of effort (and a small amount of storage) just to account 
> for C's simplistic const/free semantics.
>
> The other way, which is the one everyone uses, is to cast away the constness 
> of the pointer when passing it to free.
>
> Note that the problem arises in other contexts. That's why C++ has a category 
> of casts just for this purpose (casting away constness). Another case you 
> often see is a function in an external library that never modifies the object 
> pointed to by a parameter, but failed to declare it as const (generally 
> because it was written by someone who doesn't understand C, which is to say 
> ~97% of all C programmers).
>
> As a rule of thumb, when a function parameter itself is a pointer to a 
> const-qualified object type (this doesn't apply to const pointers to 
> non-const objects), you should be able to assume that the function does not 
> change the object. When a function parameter is a pointer to a 
> non-const-qualified aggregate type (struct or union) that happens to include 
> const-qualified members, do NOT assume that there are no cases in which those 
> objects are modified. You really have to check the implementation of that 
> function.
>
> C function contracts are weak.
>
> Now, a *good* public C library will implement proper ADTs. It will define 
> them as pointers to incomplete structures, so you can't go poking arou

[openssl-users] Const qualifier for EVP_PKEY parameters in pub_encode method

[Reinier Torenbeek]

Hi,

While implementing an engine, I run into the following prototype that I
need to implement:

struct evp_pkey_asn1_method_st
{
  
  int (*pub_encode)(X509_PUBKEY *pub, *const* EVP_PKEY *pk);
  
} /* EVP_PKEY_ASN1_METHOD */;

Inside this function, I want to assign pk to pub->pkey (and do the
necessary up and down reffing of the EVP_PKEYs involved). But that is
not allowed, because of the const qualifier on pk.

For now, I have just worked around the compiler complaints by casting to
a non-const, but I wonder whether this will bite me at some point?

Thanks,
Reinier
___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[openssl-users] Freeing of const char * fields in ASN1_OBJECT

[Reinier Torenbeek]

Hi,

The following looks wrong and actually resulted in a crash for me before
I decided to do things differently:

In asn1/asn1.h, the fields sn and ln in ASN1_OBJECT are defined as const
char *:

211 typedef struct asn1_object_st
212 {
213 const char *sn,*ln;
214 int nid;
215 int length;
216 const unsigned char *data;  /* data remains const after
init */
217 int flags;  /* Should we free this one */
218 } ASN1_OBJECT;

but in asn1/a_object.c, the get casted to void * and freed:

378 void ASN1_OBJECT_free(ASN1_OBJECT *a)
379 {
380 if (a == NULL) return;
381 if (a->flags & ASN1_OBJECT_FLAG_DYNAMIC_STRINGS)
382 {
383 #ifndef CONST_STRICT /* disable purely for compile-time strict const
checking. Doing this on a "real" compile will cause memory leaks */
384 if (a->sn != NULL) OPENSSL_free((void *)a->sn);
385 if (a->ln != NULL) OPENSSL_free((void *)a->ln);
386 #endif

Given that a lot of the code is supposed to be self-describing (due to
lack of documentation), it is somewhat disturbing that I can not rely on
the const qualifiers to be honored. Any thoughts on this?

Thanks,
Reinier

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] EVP-level load_key functions

[Reinier Torenbeek]

Hello Jakob,

Looking at crypt/store/store.h, do you agree that a store implementation
is the place where the functionality that you describe below belongs?

Thanks,
Reinier


On 8/6/15 8:44 PM, Jakob Bohm wrote:
 I think what one wants as a first approximation is
 functions that can enumerate and load keys and
 certificates from engine storage, such as smart
 cards (most engines would obviously load only
 proxy structures/handles when asked to load a
 private key).

 E.g.

PSOME_PRIVKEY_TYPE EVP_EnumPrivKeys(T hEngine, ofs_t i);

PSOME_PUBKEY_TYPE  EVP_EnumPubKeys(T hEngine, ofs_t i);

PSOME_CERT_TYPEEVP_EnumCerts(T hEngine, ofs_t i);

 Each will return either:

   The item at index i in engine storage

   NULL for no such item at index i

   (void*)(size_t)(1) for i past last currently
  valid index for such items

 Engines would have some leeway if they will return
 NULL or 1 for some values slightly past last index,
 and if there will be any relationships between indexes
 for different types.

 A second order approximation would be functions
 that can look for matching triplets:
 Given a public key, private key or certificate,
 load a matching item of one of the other 2 kinds,
 if present in storage, or return a not found
 error (of cause the cases of private key or cert
 to public key mapping are already trivial without
 calling the engine, the remaining 4 cases are the
 interesting ones, some engines can do them
 efficiently others would fall back to generic EVP
 layer loops over the first order enumeration
 functions and a generic EVP layer public key
 compare).  Note, I can imagine engines that can
 quickly go between private keys and certs, but not
 from public key to either, so all 4 cases need to
 be exposed to the engine level, even if (PrivToCert
 and CertToPriv methods have defaults that call
 PubToCert and PubToPriv).

 With these two simpler approximations, application
 code should be able to do most of the needed real
 world jobs, such as figuring out which useful
 certificates with matching private keys are present
 on an inserted smart card or key fob.

 A third item that might be wanted for some engines
 (such as the generic engine that can look in
 /etc/ssl/certs) would be a search for certs by exact
 subject distinguished name match, with the
 possibility of returning more than one result (think
 different serials/keyids).

 Arbitrary criteria searching would typically end up
 as a loop over enumeration functions anyway.

 Searching for chain building purposes can be built
 on top of all this without bloating the EVP and engine
 interfaces with all that code.

 Enjoy

 Jakob
 -- 
 Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
 Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
 This public discussion message is non-binding and may contain errors.
 WiseMo - Remote Service Management for PCs, Phones and Embedded 


 ___
 openssl-users mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] EVP-level load_key functions

[Reinier Torenbeek]

Thanks for your pointers.

I searched a bit further and noticed the existence of a STORE method and
the associated (un)register functions with an engine. Looking at its
API, it looks like it provides the key and certificate loading function
signatures that I need. Therefore, I was considering implementing (a
subset of) the STORE method functions in my engine.

From the README in crypto/store in master
https://github.com/openssl/openssl/blob/master/crypto/store/README, I
conclude that future versions will provide X509_STORE as a default
implementation for the store as well. Therefore, this seems the right
approach in the long run. I am currently using a 1.0.1 version.

However, I could not find any example of any engine implementing a
store. There do not seem to be any tests either. Can you/someone confirm
that implementing (a subset of) a store in my engine is a valid approach
at the moment (and in the future)?

Thanks,
Reinier

On 8/6/15 8:06 PM, Dr. Stephen Henson wrote:
 On Thu, Aug 06, 2015, Reinier Torenbeek wrote:

 I am interested in leveraging the following three functions:

 ENGINE_load_private_key()
 ENGINE_load_public_key()
 ENGINE_load_certificate()

 Unfortunately, the latter is missing. This was mentioned in a recent
 thread on this list as well (see
 http://www.mail-archive.com/openssl-users@openssl.org/msg77566.html). Is
 it planned for any future release?

 At some point yes but it's quite complex decided what the parameters should
 be: e.g. to lookup certificates matching one or more criteria.


 The former two functions are present though and I was looking for their
 EVP-level counterparts to access them in the proper way. Apparently,
 those do not exist either. Previously, I was under the impression that
 the EVP API exposes all engine-implemented functions. Are these
 EVP_load_xxx functions missing from EVP because they are todo in a
 future release or are they omitted by design?

 The ENGINE_load*key functions rerturn an EVP_PKEY structure which can be
 used by EVP directly.

 There are other functions which can load an EVP_PKEY structure too like the
 PEM and PKCS12 functions.

 Steve.
 --
 Dr Stephen N. Henson. OpenSSL project core developer.
 Commercial tech support now available see: http://www.openssl.org
 ___
 openssl-users mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[openssl-users] EVP-level load_key functions

[Reinier Torenbeek]

I am interested in leveraging the following three functions:

ENGINE_load_private_key()
ENGINE_load_public_key()
ENGINE_load_certificate()

Unfortunately, the latter is missing. This was mentioned in a recent
thread on this list as well (see
http://www.mail-archive.com/openssl-users@openssl.org/msg77566.html). Is
it planned for any future release?

The former two functions are present though and I was looking for their
EVP-level counterparts to access them in the proper way. Apparently,
those do not exist either. Previously, I was under the impression that
the EVP API exposes all engine-implemented functions. Are these
EVP_load_xxx functions missing from EVP because they are todo in a
future release or are they omitted by design?

By the way, I did find functions in apps/apps.c called load_key(),
load_pubkey() and load_cert(). The look quite like the EVP-level
functions I was expecting.

Thanks,
Reinier

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] How to provide KDF to ECDH key computation when using EVP API?

[Reinier Torenbeek]

Hello Jakob,
 How does this all compare to the EVP API for traditional
 DH?,  I think this is a closer equivalent for API design
 than ECDSA.
Good point. For traditional DH, no Key Derivation Function is mentioned
anywhere. It has a larger associated set of methods (see below) than
ECDH, but the compute_key() function in particular is simpler.

So this does not look too good for my case...

Reinier

struct dh_method {
const char *name;
/* Methods here */
int (*generate_key) (DH *dh);
int (*compute_key) (unsigned char *key, const BIGNUM *pub_key, DH *dh);
/* Can be null */
int (*bn_mod_exp) (const DH *dh, BIGNUM *r, const BIGNUM *a,
   const BIGNUM *p, const BIGNUM *m, BN_CTX *ctx,
   BN_MONT_CTX *m_ctx);
int (*init) (DH *dh);
int (*finish) (DH *dh);
int flags;
char *app_data;
/* If this is non-NULL, it will be used to generate parameters */
int (*generate_params) (DH *dh, int prime_len, int generator,
BN_GENCB *cb);
};

On 6/30/15 11:48 AM, Jakob Bohm wrote:
 On 28/06/2015 04:55, Reinier Torenbeek wrote:
 Hi again,

 After digging into the ECDH code a bit more, I (sort of) found an answer
 to my question.

 My reason to look at using the KDF is to apply a hash to the shared
 secret to compute a useable key within the derive function. There is a
 control value called EVP_PKEY_CTRL_MD which seems like it could be used
 for this purpose. However, for EC keys it looks like this control value
 only has a meaning for the signing functionality, not for the key
 derivation functionality. This looks like an omission to me. A small
 test showed that it would not be too hard to have the hash applied when
 doing key derivation as well.

 If the approach sketched above is not right or possible, then exposing
 the KDF function to the user of the EVP API seems a logical alternative.
 However, the KDF function prototype is rather limited, with only an in
 and out and no context at all. The latter would be required to make it
 useful.

 Since this functionality looks like it is a kind of half-finished to me,
 can anybody give some insight in its status or confirm/correct my
 conclusions?

 Thanks,
 Reinier

 On 6/19/15 4:23 PM, Reinier Torenbeek wrote:
 Hi,

 My goal is to implement ECDH in my own engine. The snippet below shows
 the struct that needs to be filled and set as the engine's ECDH method:

 struct ecdh_method {
 const char *name;
 int (*compute_key) (void *key, size_t outlen, const EC_POINT *pub_key,
 EC_KEY *ecdh, void *(*KDF) (const void *in,
 size_t inlen, void *out,
 size_t *outlen));
 # if 0
 int (*init) (EC_KEY *eckey);
 int (*finish) (EC_KEY *eckey);
 # endif
 int flags;
 char *app_data;
 };

 I intend to leverage the KDF mechanism, but it does not seem to be
 exposed in the EVP API. Is that possible at all? If yes, how do I do
 that? If no, what is the purpose of the KDF() parameter in compute_key?
 How does this all compare to the EVP API for traditional
 DH?,  I think this is a closer equivalent for API design
 than ECDSA.
 (By the way, struct ecdh_method is in crypto/ecdh/ech_locl.h, which
 seems to be a private header file. Am I supposed/allowed to include it
 anyway?)
 Unfortunately, someone thinks that OpenSSL should be made
 as useless as possible, strangely, this all began shortly
 after the heartbleed backdoor was closed.

 Enjoy

 Jakob
 -- 
 Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
 Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
 This public discussion message is non-binding and may contain errors.
 WiseMo - Remote Service Management for PCs, Phones and Embedded 


 ___
 openssl-users mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] Implementing ECDSA in an engine

[Reinier Torenbeek]

Hello Steve,

Thank you for the response. The ECDSA_set_default_method() indeed sets
up the functionality as you describe below and that seems to be good
enough for me (for now...).

Best regards,
Reinier

On 6/26/15 4:25 PM, Dr. Stephen Henson wrote:
 On Fri, Jun 26, 2015, Reinier Torenbeek wrote:

 The mechanism for implementing ECDSA in my own engine is unclear to me.
 Unfortunately, none of the example engines implement ECDSA so it is hard
 for me to find answers.

 Invoking ENGINE_set_ECDSA() does not seem to be sufficient: my setup,
 sign and verify methods never get invoked.

 There are two separate ways you can implement a public key algorithm in an
 ENGINE.

 The first is a default method which is then used for every single operation. 
 This is most suited to cryptographic accelerators.

 The second is a key specific method which is utilised for some (and maybe not
 all) operations on one key. This would be most suited for a smart card for
 example where signing might be performed by the card but verification
 performed by OpenSSL itself.

 The second form isn't well supported for ECDSA at present though you can sort
 of get that functionality using the first technique. That will be fixed
 at some point most likely in the master branch for OpenSSL 1.1.0.

 Steve.
 --
 Dr Stephen N. Henson. OpenSSL project core developer.
 Commercial tech support now available see: http://www.openssl.org
 ___
 openssl-users mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] How to provide KDF to ECDH key computation when using EVP API?

[Reinier Torenbeek]

Clarifying my intentions with a snippet of example code that I would like
to end up with (omitting some lines as well as error checking for the sake
of brevity):

ctx = EVP_PKEY_CTX_new(local_key, NULL);
EVP_PKEY_derive_init(ctx);
/* Set digest to be used to sha256 */
EVP_PKEY_CTX_ctrl(ctx, EVP_PKEY_EC,
EVP_PKEY_OP_DERIVE, EVP_PKEY_CTRL_MD, 0, (void *)EVP_sha256());
EVP_PKEY_derive_set_peer(ctx, remote_key);
/* Compute shared secret and apply sha256 to it to get a usable key */
EVP_PKEY_derive(ctx, result_key, result_len);

This is all compilable and runnable code, but the EVP_PKEY_CTX_ctrl
invocation is essentially ignored at this moment. Therefore the sha256 does
not actually take place.


On Sat, Jun 27, 2015 at 7:55 PM, Reinier Torenbeek 
reinier.torenb...@gmail.com wrote:

 Hi again,

 After digging into the ECDH code a bit more, I (sort of) found an answer
 to my question.

 My reason to look at using the KDF is to apply a hash to the shared
 secret to compute a useable key within the derive function. There is a
 control value called EVP_PKEY_CTRL_MD which seems like it could be used
 for this purpose. However, for EC keys it looks like this control value
 only has a meaning for the signing functionality, not for the key
 derivation functionality. This looks like an omission to me. A small
 test showed that it would not be too hard to have the hash applied when
 doing key derivation as well.

 If the approach sketched above is not right or possible, then exposing
 the KDF function to the user of the EVP API seems a logical alternative.
 However, the KDF function prototype is rather limited, with only an in
 and out and no context at all. The latter would be required to make it
 useful.

 Since this functionality looks like it is a kind of half-finished to me,
 can anybody give some insight in its status or confirm/correct my
 conclusions?

 Thanks,
 Reinier

 On 6/19/15 4:23 PM, Reinier Torenbeek wrote:
  Hi,
 
  My goal is to implement ECDH in my own engine. The snippet below shows
  the struct that needs to be filled and set as the engine's ECDH method:
 
  struct ecdh_method {
  const char *name;
  int (*compute_key) (void *key, size_t outlen, const EC_POINT
 *pub_key,
  EC_KEY *ecdh, void *(*KDF) (const void *in,
  size_t inlen, void
 *out,
  size_t *outlen));
  # if 0
  int (*init) (EC_KEY *eckey);
  int (*finish) (EC_KEY *eckey);
  # endif
  int flags;
  char *app_data;
  };
 
  I intend to leverage the KDF mechanism, but it does not seem to be
  exposed in the EVP API. Is that possible at all? If yes, how do I do
  that? If no, what is the purpose of the KDF() parameter in compute_key?
 
  (By the way, struct ecdh_method is in crypto/ecdh/ech_locl.h, which
  seems to be a private header file. Am I supposed/allowed to include it
  anyway?)
 
  Thanks in advance,
  Reinier
 
 



___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] How to provide KDF to ECDH key computation when using EVP API?

[Reinier Torenbeek]

Hi again,

After digging into the ECDH code a bit more, I (sort of) found an answer
to my question.

My reason to look at using the KDF is to apply a hash to the shared
secret to compute a useable key within the derive function. There is a
control value called EVP_PKEY_CTRL_MD which seems like it could be used
for this purpose. However, for EC keys it looks like this control value
only has a meaning for the signing functionality, not for the key
derivation functionality. This looks like an omission to me. A small
test showed that it would not be too hard to have the hash applied when
doing key derivation as well.

If the approach sketched above is not right or possible, then exposing
the KDF function to the user of the EVP API seems a logical alternative.
However, the KDF function prototype is rather limited, with only an in
and out and no context at all. The latter would be required to make it
useful.

Since this functionality looks like it is a kind of half-finished to me,
can anybody give some insight in its status or confirm/correct my
conclusions?

Thanks,
Reinier

On 6/19/15 4:23 PM, Reinier Torenbeek wrote:
 Hi,

 My goal is to implement ECDH in my own engine. The snippet below shows
 the struct that needs to be filled and set as the engine's ECDH method:

 struct ecdh_method {
 const char *name;
 int (*compute_key) (void *key, size_t outlen, const EC_POINT *pub_key,
 EC_KEY *ecdh, void *(*KDF) (const void *in,
 size_t inlen, void *out,
 size_t *outlen));
 # if 0
 int (*init) (EC_KEY *eckey);
 int (*finish) (EC_KEY *eckey);
 # endif
 int flags;
 char *app_data;
 };

 I intend to leverage the KDF mechanism, but it does not seem to be
 exposed in the EVP API. Is that possible at all? If yes, how do I do
 that? If no, what is the purpose of the KDF() parameter in compute_key?

 (By the way, struct ecdh_method is in crypto/ecdh/ech_locl.h, which
 seems to be a private header file. Am I supposed/allowed to include it
 anyway?)

 Thanks in advance,
 Reinier




___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

Re: [openssl-users] Implementing ECDSA in an engine

[Reinier Torenbeek]

Hi Dmitry,

Thanks for the response. I suppose I have to do the same thing as well then.

I wonder why the ECDSA_METHOD structure and the associated mechanism to
insert it into an engine is so much different from the DSA_METHOD
struct. The latter seems more straightforward to use and does not
require the whole custom PKEY to be set up. Do you know?

Best regards,
Reinier

On 6/26/15 12:51 PM, Dmitry Belyavsky wrote:
 Hello Reinier, 

 On Fri, Jun 26, 2015 at 7:47 PM, Reinier Torenbeek
 reinier.torenb...@gmail.com mailto:reinier.torenb...@gmail.com wrote:

 Hi,

 The mechanism for implementing ECDSA in my own engine is unclear
 to me.
 Unfortunately, none of the example engines implement ECDSA so it
 is hard
 for me to find answers.

 Invoking ENGINE_set_ECDSA() does not seem to be sufficient: my setup,
 sign and verify methods never get invoked. Stepping through the
 openssl
 code, it looks like I also need to register my own EVP_PKEY_EC method.
 That looks pretty complicated and I do not understand why that is
 required in the first place if I only want to replace the ECDSA
 signing
 and verification methods.

 Can someone confirm that registering my own EVP_PKEY_EC method is
 indeed
 the approach I have to take, or is there a shorter path to achieve
 my goal?


 I did not find a way shorter then provide own EVP_PKEY_METHOD. But it
 works for me this way. 

 -- 
 SY, Dmitry Belyavsky


 ___
 openssl-users mailing list
 To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[openssl-users] Implementing ECDSA in an engine

[Reinier Torenbeek]

Hi,

The mechanism for implementing ECDSA in my own engine is unclear to me.
Unfortunately, none of the example engines implement ECDSA so it is hard
for me to find answers.

Invoking ENGINE_set_ECDSA() does not seem to be sufficient: my setup,
sign and verify methods never get invoked. Stepping through the openssl
code, it looks like I also need to register my own EVP_PKEY_EC method.
That looks pretty complicated and I do not understand why that is
required in the first place if I only want to replace the ECDSA signing
and verification methods.

Can someone confirm that registering my own EVP_PKEY_EC method is indeed
the approach I have to take, or is there a shorter path to achieve my goal?

Is this the right list to ask questions about engine implementations? Or
would using openssl-dev be more appropriate?

Thanks,
Reinier

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[openssl-users] How to provide KDF to ECDH key computation when using EVP API?

[Reinier Torenbeek]

Hi,

My goal is to implement ECDH in my own engine. The snippet below shows
the struct that needs to be filled and set as the engine's ECDH method:

struct ecdh_method {
const char *name;
int (*compute_key) (void *key, size_t outlen, const EC_POINT *pub_key,
EC_KEY *ecdh, void *(*KDF) (const void *in,
size_t inlen, void *out,
size_t *outlen));
# if 0
int (*init) (EC_KEY *eckey);
int (*finish) (EC_KEY *eckey);
# endif
int flags;
char *app_data;
};

I intend to leverage the KDF mechanism, but it does not seem to be
exposed in the EVP API. Is that possible at all? If yes, how do I do
that? If no, what is the purpose of the KDF() parameter in compute_key?

(By the way, struct ecdh_method is in crypto/ecdh/ech_locl.h, which
seems to be a private header file. Am I supposed/allowed to include it
anyway?)

Thanks in advance,
Reinier

___
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users