Re: Re-arrange the library structure - Re: CMP is a subproject?
David von Oheimb wrote: > To avoid any misunderstandings on what I wrote before: > My proposal (possibly in difference to Dmitry's) was and still is *not* to > move any functionality out of the OpenSSL main repository, > but to re-arrange the library structure (likely by splitting libcrypto into > two or more libraries) to better reflect the code layering. What I observe is that the "openssl" *app* is not the best example code out there. There are a number of things, particularly those involving certificate creation, where the only obivous[%] way to get some things done is to use an openssl.conf fragment. [%]- sometimes there are non-obvious ways, but the only example code is the apps/*.c, so people wind up emulating it. There are also some things (no, I don't have a list handy) that are done by the apps code, which ought to be a library function. People wind up copy and pasting, then of course, it becomes non-obvious how to update their code. > Expected benefits: > * reduce binary code footprint in case just the crypto core or just > TLS (including crypto) is needed This last part is probably a red herring. If you link against the .a files, then you get only what you need. If you link against the .so, then in theory you pay only once for all users. > * some so far internal crypto interfaces that are used by the more > application-level code need to be exported We need to make a list of these. I think that they are more on the "poorly documented" rather than internal. > plus an actual library (say, libapps) that is more application-level and > includes everything that requires both TLS any crypto features, such as > HTTPS and part of (or even all of) apps/lib/. > This likely would provide a better pros/cons ratio than actually splitting up > libcrypto. I agree with you. I also also agree that restructuring should occur first, and I think that introducing a libapps could in the 3.x stream, but that many other things would be a 4.0 > In particular, as Tomas wrote, the openssl app will continue to provide > everything that it did before. I have advocated in the past splitting the "openssl" app into a new repo which could evolve at a different rate, and with a different level of scrutiny to the core library. -- Michael Richardson. o O ( IPv6 IøT consulting ) Sandelman Software Works Inc, Ottawa and Worldwide signature.asc Description: PGP signature
Re: Re-arrange the library structure - Re: CMP is a subproject?
nt is being started (i.e., 4.0). > > > > > > > > The openssl application could depend on that application > > > > library(ies). > > > > > > > > Tomas > > > > > > > > On Wed, 2022-07-06 at 09:32 +0200, David von Oheimb wrote: > > > > > > > > > Yes, there are number of components that should better be moved > > > > > out of the core crypto library into a more application-level > > > > > one. > > > > > As I wrote three days ago, though my email got stuck in > > > > > mailing list moderation: > > > > > > > > > > Forwarded Message > > > > > Subject: Re: CMP is a subproject? > > > > > Date: Sun, 3 Jul 2022 22:50:06 +0200 > > > > > From: David von Oheimb > > > > > To: Dmitry Belyavsky , List of openssl > > > > > committers > > > > > > > > > > Dear all, > > > > > thanks Dmitry for sharing this thought. > > > > > In a sense it is an instance of a more general suggestion I > > > > > gave > > > > > * back in 2017: Introducing an application-level library for > > > > > the CLI and OpenSSL-based applications #4992 > > > > > * and in 2020: Improve overall OpenSSL library structure > > > > > #13440 > > > > > which pertains also to CMS, HTTP, OCSP, TS, and maybe further > > > > > more application-level component(s) of libcrypto like CT. > > > > > The CMP implementation does not rely on libssl, but it does > > > > > heavily rely on libcrypto and relies on some of its internals. > > > > > The same holds for HTTP, and likely this also holds for CMS, > > > > > OCSP, TS, and CT. > > > > > David > > > > > > > > > > > > > > On 06.07.22 07:25, Dr Paul Dale wrote: > > > > > I'd support such a change. Our stability policy won't without > > > > > an exception. There are a lot more things that could be moved > > > > > out IMO. Pauli On 6/7/22 15:22, Benjamin Kaduk wrote: > > > > > > > > > > > On Sun, Jul 03, 2022 at 09:51:23PM +0200, Dmitry Belyavsky > > > > > > wrote: > > > > > > > > > > > > > Dear colleagues, With all respect to David's efforts - > > > > > > > isn't it worth turning CMP into a separate library in > > > > > > > OpenSSL (and probably into a separate repo)? I remember > > > > > > > there was a separate PR in this direction. > > > > > > I think I found > > > > > > https://github.com/openssl/openssl/issues/16358 just now, but > > > > > > maybe there are others. > > > > > > > > > > > > > It looks like CMP heavily relies on libcrypto/libssl, but > > > > > > > I'm not sure it requires an integration - and, last but not > > > > > > > least, has its own life cycle. Several years ago this > > > > > > > seemed a good rationale both to me and to the OpenSSL team > > > > > > > to separate a GOST engine. > > > > > > It looks like there was some discussion in > > > > > > https://github.com/openssl/openssl/pull/6811 that suggests > > > > > > that having apps/cmp.c functionality was a key motivation for > > > > > > pulling in everything to libcrypto itself, but I'm not sure > > > > > > how far the conversation of in-OpenSSL vs standalond project > > > > > > really went at that time. I don't think I have anything to > > > > > > add to that discussion other than what you say above. -Ben > > > > > > -- > Tomáš Mráz, OpenSSL > >
Re: Re-arrange the library structure - Re: CMP is a subproject?
Just moving things around in the source tree does not achieve much so without the actual splitting the functionality out of the libcrypto does not make sense to me. Maybe it could be seen as a preparation step for the split out. However yes, it was a misunderstanding IMO that we would want to split it completely out of the main OpenSSL project tree. I do not think this is necessary or even desirable. However as many applications do not need this functionality it would be useful to have it as a separate shared library so loading libcrypto is less expensive. Still I think this is a 4.0 project. Tomas On Fri, 2022-07-08 at 09:00 +0200, David von Oheimb wrote: > On 07.07.22 23:02, Tim Hudson wrote: > > > I do not think this makes sense at this stage at all. > > One of the key elements people are looking for when contributing > > code is the distribution vector of getting including in default OS > > distributions and standard builds. > I fully agree. > To avoid any misunderstandings on what I wrote before: > My proposal (possibly in difference to Dmitry's) was and still is > not to move any functionality out of the OpenSSL main repository, > but to re-arrange the library structure (likely by splitting > libcrypto into two or more libraries) to better reflect the code > layering. > Expected benefits: > * improve clarity of the software component structure > * slightly alleviate development and maintenance > * reduce binary code footprint in case just the crypto core or just > TLS (including crypto) is needed > Expected drawbacks: > * any re-structuring requires more or less work > * some so far internal crypto interfaces that are used by the more > application-level code need to be exported > * applications that also require the more high-level capabilities > will need to link with more libraries > We may also consider splitting the existing libcrypto just virtually, > e.g., into two code directories (say, crypto/ and crypto/apps/, which > includes CMS, CMP, OCSP, HTTP, TS, etc.) > plus an actual library (say, libapps) that is more application-level > and includes everything that requires both TLS any crypto features, > such as HTTPS and part of (or even all of) apps/lib/. > This likely would provide a better pros/cons ratio than actually > splitting up libcrypto. > > > > This is something we could look at tackling in a future major > > release - but even then it faces challenges to be workable for the > > desired outcome (broad distribution of capability). > With just re-arranging the code into three (or more, rather than so > far two) OpenSSL libraries, there will be no issue with capability > because nothing is lost for OpenSSL users. > In particular, as Tomas wrote, the openssl app will continue to > provide everything that it did before. > > David > > > > On Thu, 7 July 2022, 18:48 Tomas Mraz, wrote: > > > > > OpenSSL Project list should be used instead of the committers > > > list for > > > such discussions. > > > > > > I do not think it would be good idea to do any such splitting > > > before a > > > major release development is being started (i.e., 4.0). > > > > > > The openssl application could depend on that application > > > library(ies). > > > > > > Tomas > > > > > > On Wed, 2022-07-06 at 09:32 +0200, David von Oheimb wrote: > > > > > > > Yes, there are number of components that should better be moved > > > > out of the core crypto library into a more application-level > > > > one. > > > > As I wrote three days ago, though my email got stuck in > > > > mailing list moderation: > > > > > > > > Forwarded Message > > > > Subject: Re: CMP is a subproject? > > > > Date: Sun, 3 Jul 2022 22:50:06 +0200 > > > > From: David von Oheimb > > > > To: Dmitry Belyavsky , List of openssl > > > > committers > > > > > > > > Dear all, > > > > thanks Dmitry for sharing this thought. > > > > In a sense it is an instance of a more general suggestion I > > > > gave > > > > * back in 2017: Introducing an application-level library for > > > > the CLI and OpenSSL-based applications #4992 > > > > * and in 2020: Improve overall OpenSSL library structure > > > > #13440 > > > > which pertains also to CMS, HTTP, OCSP, TS, and maybe further > > > > more application-level component(s) of libcrypto like CT. > > > > The CMP
Re-arrange the library structure - Re: CMP is a subproject?
On 07.07.22 23:02, Tim Hudson wrote: I do not think this makes sense at this stage at all. One of the key elements people are looking for when contributing code is the distribution vector of getting including in default OS distributions and standard builds. I fully agree. To avoid any misunderstandings on what I wrote before: My proposal (possibly in difference to Dmitry's) was and still is *not* to move any functionality out of the OpenSSL main repository, but to re-arrange the library structure (likely by splitting libcrypto into two or more libraries) to better reflect the code layering. Expected benefits: * improve clarity of the software component structure * slightly alleviate development and maintenance * reduce binary code footprint in case just the crypto core or just TLS (including crypto) is needed Expected drawbacks: * any re-structuring requires more or less work * some so far internal crypto interfaces that are used by the more application-level code need to be exported * applications that also require the more high-level capabilities will need to link with more libraries We may also consider splitting the existing libcrypto just virtually, e.g., into two code directories (say, crypto/ and crypto/apps/, which includes CMS, CMP, OCSP, HTTP, TS, etc.) plus an actual library (say, libapps) that is more application-level and includes everything that requires both TLS any crypto features, such as HTTPS and part of (or even all of) apps/lib/. This likely would provide a better pros/cons ratio than actually splitting up libcrypto. This is something we could look at tackling in a future major release - but even then it faces challenges to be workable for the desired outcome (broad distribution of capability). With just re-arranging the code into three (or more, rather than so far two) OpenSSL libraries, there will be no issue with capability because nothing is lost for OpenSSL users. In particular, as Tomas wrote, the openssl app will continue to provide everything that it did before. David On Thu, 7 July 2022, 18:48 Tomas Mraz, wrote: OpenSSL Project list should be used instead of the committers list for such discussions. I do not think it would be good idea to do any such splitting before a major release development is being started (i.e., 4.0). The openssl application could depend on that application library(ies). Tomas On Wed, 2022-07-06 at 09:32 +0200, David von Oheimb wrote: Yes, there are number of components that should better be moved out of the core crypto library into a more application-level one. As I wrote three days ago, though my email got stuck in mailing list moderation: Forwarded Message Subject: Re: CMP is a subproject? Date: Sun, 3 Jul 2022 22:50:06 +0200 From: David von Oheimb To: Dmitry Belyavsky , List of openssl committers Dear all, thanks Dmitry for sharing this thought. In a sense it is an instance of a more general suggestion I gave * back in 2017: Introducing an application-level library for the CLI and OpenSSL-based applications #4992 <https://github.com/openssl/openssl/pull/4992> * and in 2020: Improve overall OpenSSL library structure #13440 <https://github.com/openssl/openssl/issues/13440> which pertains also to CMS, HTTP, OCSP, TS, and maybe further more application-level component(s) of libcrypto like CT. The CMP implementation does not rely on libssl, but it does heavily rely on libcrypto and relies on some of its internals. The same holds for HTTP, and likely this also holds for CMS, OCSP, TS, and CT. David On 06.07.22 07:25, Dr Paul Dale wrote: I'd support such a change. Our stability policy won't without an exception. There are a lot more things that could be moved out IMO. Pauli On 6/7/22 15:22, Benjamin Kaduk wrote: On Sun, Jul 03, 2022 at 09:51:23PM +0200, Dmitry Belyavsky wrote: Dear colleagues, With all respect to David's efforts - isn't it worth turning CMP into a separate library in OpenSSL (and probably into a separate repo)? I remember there was a separate PR in this direction. I think I foundhttps://github.com/openssl/openssl/issues/16358 just now, but maybe there are others. It looks like CMP heavily relies on libcrypto/libssl, but I'm not sure it requires an integration - and, last but not least, has its own life cycle. Several years ago this seemed a good rationale both to me and to the OpenSSL team to separate a GOST engine. It looks like there was some discussion in https://github.com/openssl/openssl/pull/6811 that suggests that having apps/cmp.c functionality was a key motivation for pulling in everything to libcrypto itself, but I'm not sure how far the conve
Re: CMP is a subproject?
I do not think this makes sense at this stage at all. One of the key elements people are looking for when contributing code is the distribution vector of getting including in default OS distributions and standard builds. This is something we could look at tackling in a future major release - but even then it faces challenges to be workable for the desired outcome (broad distribution of capability). Tim. On Thu, 7 July 2022, 18:48 Tomas Mraz, wrote: > OpenSSL Project list should be used instead of the committers list for > such discussions. > > I do not think it would be good idea to do any such splitting before a > major release development is being started (i.e., 4.0). > > The openssl application could depend on that application library(ies). > > Tomas > > On Wed, 2022-07-06 at 09:32 +0200, David von Oheimb wrote: > > Yes, there are number of components that should better be moved out > > of the core crypto library into a more application-level one. > > As I wrote three days ago, though my email got stuck in mailing list > > moderation: > > > > ---- Forwarded Message Subject: Re: CMP is a > > subproject? Date: Sun, 3 Jul 2022 22:50:06 +0200 From: David von > > Oheimb To: Dmitry Belyavsky > > , List of openssl committers > > > > Dear all, thanks Dmitry for sharing this thought. > > In a sense it is an instance of a more general suggestion I gave > > * back in 2017: Introducing an application-level library for the > > CLI and OpenSSL-based applications #4992 > > * and in 2020: Improve overall OpenSSL library structure #13440 > > which pertains also to CMS, HTTP, OCSP, TS, and maybe further more > > application-level component(s) of libcrypto like CT. > > The CMP implementation does not rely on libssl, but it does heavily > > rely on libcrypto and relies on some of its internals. > > The same holds for HTTP, and likely this also holds for CMS, OCSP, > > TS, and CT. > > David > > > > > > On 06.07.22 07:25, Dr Paul Dale wrote: > > > > > I'd support such a change. Our stability policy won't without an > > > exception. > > > > > > There are a lot more things that could be moved out IMO. > > > > > > > > > Pauli > > > > > > > > > On 6/7/22 15:22, Benjamin Kaduk wrote: > > > > > > > On Sun, Jul 03, 2022 at 09:51:23PM +0200, Dmitry Belyavsky wrote: > > > > > > > > > Dear colleagues, > > > > > > > > > > With all respect to David's efforts - isn't it worth turning > > > > > CMP into a > > > > > separate library in OpenSSL (and probably into a separate > > > > > repo)? I remember > > > > > there was a separate PR in this direction. > > > > I think I found https://github.com/openssl/openssl/issues/16358 > > > > just now, > > > > but maybe there are others. > > > > > > > > > > > > > It looks like CMP heavily relies on libcrypto/libssl, but I'm > > > > > not sure it > > > > > requires an integration - and, last but not least, has its own > > > > > life cycle. > > > > > Several years ago this seemed a good rationale both to me and > > > > > to the > > > > > OpenSSL team to separate a GOST engine. > > > > It looks like there was some discussion in > > > > https://github.com/openssl/openssl/pull/6811 that suggests that > > > > having > > > > apps/cmp.c functionality was a key motivation for pulling in > > > > everything to > > > > libcrypto itself, but I'm not sure how far the conversation of > > > > in-OpenSSL > > > > vs standalond project really went at that time. I don't think I > > > > have > > > > anything to add to that discussion other than what you say above. > > > > > > > > -Ben > > > > > > -- > Tomáš Mráz, OpenSSL > >
Re: CMP is a subproject?
OpenSSL Project list should be used instead of the committers list for such discussions. I do not think it would be good idea to do any such splitting before a major release development is being started (i.e., 4.0). The openssl application could depend on that application library(ies). Tomas On Wed, 2022-07-06 at 09:32 +0200, David von Oheimb wrote: > Yes, there are number of components that should better be moved out > of the core crypto library into a more application-level one. > As I wrote three days ago, though my email got stuck in mailing list > moderation: > > Forwarded Message -------- Subject: Re: CMP is a > subproject? Date: Sun, 3 Jul 2022 22:50:06 +0200 From: David von > Oheimb To: Dmitry Belyavsky > , List of openssl committers > > Dear all, thanks Dmitry for sharing this thought. > In a sense it is an instance of a more general suggestion I gave > * back in 2017: Introducing an application-level library for the > CLI and OpenSSL-based applications #4992 > * and in 2020: Improve overall OpenSSL library structure #13440 > which pertains also to CMS, HTTP, OCSP, TS, and maybe further more > application-level component(s) of libcrypto like CT. > The CMP implementation does not rely on libssl, but it does heavily > rely on libcrypto and relies on some of its internals. > The same holds for HTTP, and likely this also holds for CMS, OCSP, > TS, and CT. > David > > > On 06.07.22 07:25, Dr Paul Dale wrote: > > > I'd support such a change. Our stability policy won't without an > > exception. > > > > There are a lot more things that could be moved out IMO. > > > > > > Pauli > > > > > > On 6/7/22 15:22, Benjamin Kaduk wrote: > > > > > On Sun, Jul 03, 2022 at 09:51:23PM +0200, Dmitry Belyavsky wrote: > > > > > > > Dear colleagues, > > > > > > > > With all respect to David's efforts - isn't it worth turning > > > > CMP into a > > > > separate library in OpenSSL (and probably into a separate > > > > repo)? I remember > > > > there was a separate PR in this direction. > > > I think I found https://github.com/openssl/openssl/issues/16358 > > > just now, > > > but maybe there are others. > > > > > > > > > > It looks like CMP heavily relies on libcrypto/libssl, but I'm > > > > not sure it > > > > requires an integration - and, last but not least, has its own > > > > life cycle. > > > > Several years ago this seemed a good rationale both to me and > > > > to the > > > > OpenSSL team to separate a GOST engine. > > > It looks like there was some discussion in > > > https://github.com/openssl/openssl/pull/6811 that suggests that > > > having > > > apps/cmp.c functionality was a key motivation for pulling in > > > everything to > > > libcrypto itself, but I'm not sure how far the conversation of > > > in-OpenSSL > > > vs standalond project really went at that time. I don't think I > > > have > > > anything to add to that discussion other than what you say above. > > > > > > -Ben > > > -- Tomáš Mráz, OpenSSL
