Re: Memory leak in openssl 1.1.1d
This isn’t enough information to diagnose the issue. Which of the leak summary records is the problem? Are you sure that your application is cleaning up properly (hint: it isn’t, e.g. OpenSSL never calls operator new() from the second record). Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 10 Sep 2020, at 1:39 am, Abhi Arora wrote: > > I am using openssl 1.1.1d. I found out around 228 bytes are being directly > lost (as per valgrind) report. I have one application which uses curl (7.64) > and I wrote the same application using POCO HTTPS and I got the same result. > > I thought it could be related to the cipher suit. I can see the leak is > dependent on the URL being used (all were https). For one of the urls, it was > leaking 228 (directly) per connection and disconnection. For one of the urls, > the same code doesn't leak atall (no matter how many times you run that code > in a for loop). > > For the urls which were leaking, the following cipher suites were being used: > 1. SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 > 2. SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 > > Here is the valgrind summary: > ==8620== HEAP SUMMARY: > ==8620== in use at exit: 121,777 bytes in 1,097 blocks > ==8620== total heap usage: 129,382 allocs, 128,285 frees, 10,635,842 bytes > allocated > ==8620== > ==8620== 8 bytes in 1 blocks are possibly lost in loss record 16 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4E75523: ??? (in /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 8 bytes in 1 blocks are definitely lost in loss record 17 of 372 > ==8620==at 0x483F310: operator new(unsigned int) (vg_replace_malloc.c:336) > ==8620==by 0x36D15: main (sample_pki.cpp:27) > ==8620== > ==8620== 8 bytes in 1 blocks are definitely lost in loss record 18 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4CBF14B: CRYPTO_zalloc (mem.c:230) > ==8620==by 0x4C76E07: ECDSA_SIG_new (ec_asn1.c:1214) > ==8620==by 0x537064B: pkcs11_try_pkey_ec_sign (p11_pkey.c:546) > ==8620==by 0x537064B: pkcs11_pkey_ec_sign (p11_pkey.c:634) > ==8620==by 0x4CB70CD: EVP_DigestSignFinal (m_sigver.c:148) > ==8620==by 0x4BA9FF3: tls_construct_cert_verify (statem_lib.c:307) > ==8620==by 0x4BA42C7: write_state_machine (statem.c:843) > ==8620==by 0x4BA42C7: state_machine (statem.c:443) > ==8620==by 0x4B8BABB: ssl3_read_bytes (rec_layer_s3.c:1278) > ==8620==by 0x4B900E1: ssl3_read_internal (s3_lib.c:4473) > ==8620==by 0x4B9013F: ssl3_read (s3_lib.c:4497) > ==8620==by 0x4B96A6D: ssl_read_internal (ssl_lib.c:1761) > ==8620==by 0x4B96B09: SSL_read (ssl_lib.c:1775) > ==8620== > ==8620== 16 bytes in 2 blocks are possibly lost in loss record 55 of 372 > ==8620==at 0x483EA30: malloc (vg_replace_malloc.c:306) > ==8620==by 0x48419FF: realloc (vg_replace_malloc.c:834) > ==8620==by 0x4E73B5D: amqpvalue_set_map_value (in > /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 40 bytes in 10 blocks are possibly lost in loss record 182 of 372 > ==8620==at 0x483EA30: malloc (vg_replace_malloc.c:306) > ==8620==by 0x48419FF: realloc (vg_replace_malloc.c:834) > ==8620==by 0x4E738BB: amqpvalue_set_list_item (in > /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 48 bytes in 1 blocks are possibly lost in loss record 210 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4E7582D: ??? (in /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 48 bytes in 2 blocks are possibly lost in loss record 211 of 372 > ==8620==at 0x48419B0: realloc (vg_replace_malloc.c:834) > ==8620==by 0x4E73B5D: amqpvalue_set_map_value (in > /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 56 bytes in 1 blocks are possibly lost in loss record 215 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4E74FF9: ??? (in /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 64 bytes in 2 blocks are possibly lost in loss record 228 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4E74477: ??? (in /usr/lib/libuamqp.so.1.2.12) > ==8620== > ==8620== 64 bytes in 1 blocks are definitely lost in loss record 229 of 372 > ==8620==at 0x483EB28: malloc (vg_replace_malloc.c:307) > ==8620==by 0x4A8D411: __libc_alloc_buffer_allocate > (alloc_buffer_allocate.c:26) > ==8620==by 0x4AE2B51: alloc_buffer_allocate (alloc_buffer.h:143) > ==8620==by 0x4AE2B51: __resolv_conf_allocate (resolv_conf.c:411) > ==8620==by 0x4AE147D: __resolv_conf_load (res_init.c:592) > ==8620==by
Re: Integration of new algorithms
Instead of using an engine, you should write a provider (assuming you’re using the soon to be released OpenSSL 3.0). It doesn’t need a NID. If you are using OpenSSL 1.1.1, try the OBJ_new_nid() function. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 26 Aug 2020, at 6:48 pm, Kris Kwiatkowski wrote: > > > Hey, > > I'm working on development of OpenSSL ENGINE that integrates > post-quantum algorithms (new NIDs). During integration I > need to modify OpenSSL code to add custom function, but would > prefer not to need add anything to OpenSSL code (so engine > can be dynmicaly loaded by any modern OpenSSL). > > So, In three cases, namely when the code is in callbacks for keygen, > encryption and ctrl (called by EVP_PKEY_CTX_ctrl, EVP_PKEY_encrypt > and EVP_PKEY_keygen) I need to get NID of the scheme. The problem > is that, those functions are called with EVP_PKEY_CTX object > provided as an argument. The NID is stored in the > EVP_PKEY_CTX->pmeth->pkey_id. I think (AFAIK) there is no API > which would return that value. > > I've added a simple function that returns pkey_id from the ctx, but > that means that I need to change OpenSSL code. Is there any way > to get NID without changing OpenSSL? > > Kind regards, > Kris > > > >
Re: VOTE: Accept the OTC voting policy as defined:
+1 Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 28 Sep 2020, at 10:02 pm, Dr. Matthias St. Pierre > wrote: > > topic: Accept the OTC voting policy as defined: > > The proposer of a vote is ultimately responsible for updating the > votes.txt > file in the repository. Outside of a face to face meeting, voters MUST > reply > to the vote email indicating their preference and optionally their > reasoning. > Voters MAY update the votes.txt file in addition. > > The proposed vote text SHOULD be raised for discussion before calling > the vote. > > Public votes MUST be called on the project list, not the OTC list and > the > subject MUST begin with "VOTE:". Private votes MUST be called on the > OTC list with "PRIVATE VOTE:" beginning subject. > > Proposed by Matthias St. Pierre (on behalf of the OTC) > Public: yes > opened: 2020-09-28 > closed: 2020-mm-dd > accepted: yes/no (for: X, against: Y, abstained: Z, not voted: T) > > Matt [ ] > Mark [ ] > Pauli [ ] > Viktor [ ] > Tim[ ] > Richard[ ] > Shane [ ] > Tomas [ ] > Kurt [ ] > Matthias [+1] > Nicola [ ] >
Freeze?
I’m seeing quite a bit of activity going on which isn’t related to the 3.0beta1 milestone. We’re well past the cutoff date announced for new features in the code. Should we be limiting the “new” stuff going in? I’m fine with bug fixes, they make sense. I’m fine with the list of beta1 pull requests continuing. It’s the rest that is more concerning. Does anyone else have a similar view? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: New GitHub label for release blockers
> BTW: It took me all my force of will to resist the temptation of making a pun > by naming the label [urgent: beta blocker]. Failed you have. Your training is not yet compete :) Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Beta1 PR deadline
Shane has done the IG D.9 work: it is in #12835 and is awaiting review. I think it should be in beta if possible. It’s mostly adding a second test case due to NIST changing guidance recently, so low risk and helpful for the lab. As Shane notes in 12801, there will be some conflicts generated over the FIPS error handling and self tests, hopefully they won’t slow progress. 12754 could be considered optional and something better should likely be done for 3.1 that what I’m planning there. With RAND_DRBG removed, there is nothing remotely comparable. Even RAND_DRBG is a bit of a stretch as a comparison. The API renaming discussion *must* reach a conclusion before beta. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 10 Sep 2020, at 8:40 pm, Matt Caswell wrote: > > > > On 09/09/2020 13:03, Kurt Roeckx wrote: >> On Wed, Aug 26, 2020 at 04:58:26PM +0100, Matt Caswell wrote: >>> Please can anyone with PRs that they wish to have included in OpenSSL >>> 3.0 beta1 ensure that they are merged to master by 8th September. >> >> So that date has passed now. Can someone give an overview of what >> we think is still needed to be done before the beta 1 release? Are >> there open PRs for everything? Do we a milestone on github to >> indicate which PRs need to go in before beta1? > > > I believe the "blockers" to be: > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12536__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JQc5z_l4$ > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12754__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JseFWMvQ$ > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12750__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J0OAUQ5s$ > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12781__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-JYPbPTZA$ > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/12801__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J_mL-S18$ > > > Aside from these there are potential PRs not yet opened but slated for > beta1 regarding: > - Refactor serialization? (owner: Richard) > - lhash refactor? (owner: Me) > - IG D.9 update? (owner: Shane) > > There's also this issue - but I'm not sure that it needs to be resolved > for beta1: > https://urldefense.com/v3/__https://github.com/openssl/openssl/issues/12195__;!!GqivPVa7Brio!O5gzpHuWIANvRtRYOHA9m7SK4SqMUb2uLzEGQbUrX9nFSRhT7-sA0F-J1-VRwBk$ > > > Matt
Re: More GitHub labels
Matthias, Your suggestion seems workable too. PRs are merged with outstanding change requests indicated — a reviewer comments, the comments are addressed then a different reviewer approves without the original review being removed. The labels are a bit more in your face. A hybrid “hold: required changes see review/comments” might be workable. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 10 Sep 2020, at 4:03 pm, Dr. Matthias St. Pierre > wrote: > >> Just wondering if we should have two new labels: “hold: tests needed” and >> “hold: documentation needed” labels? >> There are a number of PRs that come through where one or both of these are >> missing missing. > > The two use cases you mention are actually better handled by a change request > (via GitHub review) instead of a label: > A team member can formulate a change request to block the merging. Here he > has more freedom to formulate what > needs to be done. This includes your two use cases, as well as the use case > for the label [hold: need rebase]. > > The problem with it is that currently we count only approvals and don’t > consider unresolved change requests > as a blocker. I think we should change that. This does not mean that a > reviewer who made a change request > two months ago and lost interest is forced to re-review, only that such stale > reviews must be dismissed > explicitly, if the reviewer does not respond to a re-review request within a > certain time period. > > Matthias >
More GitHub labels
Just wondering if we should have two new labels: “hold: tests needed” and “hold: documentation needed” labels? There are a number of PRs that come through where one or both of these are missing (this post posed by @slontis’s comment in 12826 <https://github.com/openssl/openssl/pull/12826#pullrequestreview-485480847>). These would give a clear indication of what additional material is being expected as per CLA required and need rebase. A “hold: code needed" seems less useful :) Thoughts or should we just add them? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Reordering new API's that have a libctx, propq
Still no need for the added complexity: Push: OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx); Pop is: OPENSSL_CTX_set0_default(prevctx); Push before callback: OPENSSL_CTX_set0_default(prevctx); Pop after callback: prevctx = OPENSSL_CTX_set0_default(libctx); or OPENSSL_CTX_set0_default(libctx); Depending if we want to support call backs changing the Libctx directly — we should choose one and always recommend that. Also the auto allocation of storage for the second on stack cannot fail, so no error checking spaghetti. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 10 Sep 2020, at 12:08 am, Tomas Mraz wrote: > > On Wed, 2020-09-09 at 22:29 +1000, Dr Paul Dale wrote: >>> On 9 Sep 2020, at 9:38 pm, Tomas Mraz wrote: >>> >>> We could even provide a convenience thread local stack of lib >>> contexts >>> so the caller would not have to keep the old value but would just >>> push >>> the new libctx when entering and pop the old one when leaving. With >>> that, I think the changes needed in the application code would be >>> fairly simple and minimal. >> >> Let’s not overcomplicate things. >> We went through this discussion back when this was introduced. >> >> >> Push is: >>OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx); >> >> Pop is: >>OPENSSL_CTX_set0_default(prevctx) >> >> >> I don’t see having an explicit stack of these is of any benefit to >> anything but unwarranted complexity. > > There is one thing where it would be IMO helpful - let's say libcurl > has a callback into a calling application. With the current API in > libcurl API calls you would put the > calls OPENSSL_CTX_set0_default(libctx)/OPENSSL_CTX_set0_default(prevctx > ) at the beginning and end. But you would have to save the prevctx > somewhere in the libcurl context structure because on callbacks you > would have to temoprarily reset the context to the prevctx value. If we > implemented real stack it would not be needed. But yeah, I am not sure > this convenience is that much worth it. > > -- > Tomáš Mráz > No matter how far down the wrong road you've gone, turn back. > Turkish proverb > [You'll know whether the road is wrong if you carefully listen to your > conscience.]
Re: Reordering new API's that have a libctx, propq
> On 9 Sep 2020, at 9:38 pm, Tomas Mraz wrote: > > We could even provide a convenience thread local stack of lib contexts > so the caller would not have to keep the old value but would just push > the new libctx when entering and pop the old one when leaving. With > that, I think the changes needed in the application code would be > fairly simple and minimal. Let’s not overcomplicate things. We went through this discussion back when this was introduced. Push is: OPENSSL_CTX *prevctx = OPENSSL_CTX_set0_default(libctx); Pop is: OPENSSL_CTX_set0_default(prevctx) I don’t see having an explicit stack of these is of any benefit to anything but unwarranted complexity. Pauli
Re: Beta1 PR deadline
It is also worth noting that new features will not be accepted during the beta period. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 27 Aug 2020, at 1:58 am, Matt Caswell wrote: > > Hi all > > The OMC had a meeting today. > > Please can anyone with PRs that they wish to have included in OpenSSL > 3.0 beta1 ensure that they are merged to master by 8th September. > > Note in particular that there is no PR at the moment to incorporate SM2 > asymmetric encryption into OpenSSL 3.0. This feature currently exists in > 1.1.1, so if no PR emerges by the above date then this feature is liable > to be dropped in 3.0. (Note: PRs for SM2 signatures *do* exist and are > expected to be present). > > > Matt > >
TLS 1.3 illustrated
This might be interesting to some: https://tls13.ulfheim.net <https://tls13.ulfheim.net/> Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
RAND_DRBG futures
I’ve closed the vote. Five for, none against, the vote passes. RAND_DRBG will be absent in 3.0. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: RAND_DRBG
So far a universal voice for removal of the DRBG_RAND APIs. I’ll write up an OMC vote. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 27 Jul 2020, at 6:51 pm, Matt Caswell wrote: > > I'm ok with option 1 (but it will require a vote). I think the > percentage of our user base that are using the existing API is > sufficiently close to zero that we're not breaking our compatibility > promises. > > Matt > > > On 27/07/2020 02:08, Dr Paul Dale wrote: >> The RAND_DRBG (crypto/rand/drbg_lib) APIs are quite some mess and sit >> badly with the move to provider based infrastructure. >> They are definitely being deprecated in master but without more, the >> extra layer of indirection and additional complexity generating random >> numbers will remain. >> >> The option I see are: >> >> 1. Remove, rather than deprecate, RAND_DRBG in 3.0. This is a breaking >> change. >> 2. Bypass RAND_DRBG and live with a breaking change (refer: >> https://github.com/openssl/openssl/pull/12509#pullrequestreview-455396828) >> 3. Leave things as they currently are in master. >> >> The first two are breaking changes and will require an OMC vote. >> >> >> Some pertinent points: >> >> 1. RAND_bytes will continue to work as is — this API is perfect for >> almost everyone. >> 2. The RAND_METHOD functionality remains — this allows wholesale >> replacement of OpenSSL’s RNGs if desired. >> 3. The name EVP_RAND is the working name and might change — this is not >> relevant to this discussion. >> 4. The RAND_DRBG APIs are unlikely to be widely used — they were >> introduced in 1.1.1. The two users I know of (Akamai and NCP) are both >> fine with them being removed. >> >> >> Thoughts anyone? >> >> >> Pauli >> -- >> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >> Phone +61 7 3031 7217 >> Oracle Australia >>
RAND_DRBG
The RAND_DRBG (crypto/rand/drbg_lib) APIs are quite some mess and sit badly with the move to provider based infrastructure. They are definitely being deprecated in master but without more, the extra layer of indirection and additional complexity generating random numbers will remain. The option I see are: 1. Remove, rather than deprecate, RAND_DRBG in 3.0. This is a breaking change. 2. Bypass RAND_DRBG and live with a breaking change (refer: https://github.com/openssl/openssl/pull/12509#pullrequestreview-455396828) 3. Leave things as they currently are in master. The first two are breaking changes and will require an OMC vote. Some pertinent points: 1. RAND_bytes will continue to work as is — this API is perfect for almost everyone. 2. The RAND_METHOD functionality remains — this allows wholesale replacement of OpenSSL’s RNGs if desired. 3. The name EVP_RAND is the working name and might change — this is not relevant to this discussion. 4. The RAND_DRBG APIs are unlikely to be widely used — they were introduced in 1.1.1. The two users I know of (Akamai and NCP) are both fine with them being removed. Thoughts anyone? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: API renaming
I think the types should change to match any function name changes. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Jul 2020, at 2:45 am, Short, Todd wrote: > > They also correspond directly to EVP_MAC and EVP_KDF types. Would the types > change as well? > -- > -Todd Short > // tsh...@akamai.com <mailto:tsh...@akamai.com> > // “One if by land, two if by sea, three if by the Internet." > >> On Jul 23, 2020, at 11:56 AM, Matt Caswell > <mailto:m...@openssl.org>> wrote: >> >> >> >> On 23/07/2020 16:52, Richard Levitte wrote: >>> On Thu, 23 Jul 2020 12:18:10 +0200, >>> Dr Paul Dale wrote: >>>> There has been a suggestion to rename EVP_RAND to OSSL_RAND. This seems >>>> reasonable. Would it >>>> also make sense to rename the other new APIs similarly. >>>> More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF >>>> respectively? >>> >>> This is a good question... >>> >>> Historically speaking, even though EVP_MAC and EVP_KDF are indeed new >>> APIs, they have a previous history of EVP APIs, through EVP_PKEY. The >>> impact of relocating them outside of the EVP "family" may be small, >>> but still, history gives me pause. >>> >>> RAND doesn't carry the same sort of history, which makes it much >>> easier for me to think "just do it and get it over with"... >> >> I have the same pause - so I'm thinking just RAND for now. >> >> Matt >> >
Re: API renaming
Adherence to the code style will also be required (indentation will change). This will be harder to automate. Changing EVP_RAND -> OSSL_RAND is worse because it will change line breaks as well as indentation. OSSL_RNG avoids this, if we accept not using RAND in the name. KDF and MAC also get longer and would have the same issue. Then there are the inevitable merge conflicts…. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Jul 2020, at 6:15 pm, Dr. Matthias St. Pierre > wrote: > >> I was thinking OSSL_LIBCTX? > > That's a good choice and consistent with how we name the variable in most > (but not all) places: > > OPENSSL_CTX *libctx; > > I volunteer to raise a pull request which does a scripted bulk rename, as > soon as we have made > a decision. Ideally, the bulk renaming should go in shortly before the next > alpha. Having it > automated by a script would ease rebasing of other still unmerged pull > requests over the change. > > Matthias > >> -Original Message- >> From: SHANE LONTIS >> Sent: Friday, July 24, 2020 9:43 AM >> To: Dr. Matthias St. Pierre >> Cc: Richard Levitte ; openssl-project@openssl.org >> Subject: Re: API renaming >> >> I was thinking OSSL_LIBCTX? >> >
Re: API renaming
The exact same points apply to EVP_MAC & EVP_KDF. We have also been telling people “use EVP” for ages. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Jul 2020, at 3:20 pm, Richard Levitte wrote: > > A couple of points: > > 1. Quite a while ago, we (the team at the time) made a decision to >have all new APIs prefixed with 'OPENSSL_' or 'OSSL_'. It seems >that we never voted on it, though, but still. > > 2. The new RAND API hasn't been merged yet, so it's not like we're >renaming something that already exists. > > So in terms of "it's just a prefix", OSSL_ would be just as suitable. > It's a bit more blatantly "OpenSSL", though. > > Cheers, > Richard > > On Thu, 23 Jul 2020 23:30:25 +0200, > Tim Hudson wrote: >> Placing everything under EVP is reasonable in my view. It is just a prefix >> and it really has no >> meaning these days as it became nothing more than a common prefix to use. >> >> I don't see any significant benefit in renaming at this point - even for >> RAND. >> >> Tim. >> >> On Fri, 24 Jul 2020, 1:56 am Matt Caswell, wrote: >> >>On 23/07/2020 16:52, Richard Levitte wrote: >>> On Thu, 23 Jul 2020 12:18:10 +0200, >>> Dr Paul Dale wrote: >>>> There has been a suggestion to rename EVP_RAND to OSSL_RAND. This seems >>>> reasonable. Would >>it >>>> also make sense to rename the other new APIs similarly. >>>> More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF >>>> respectively? >>> >>> This is a good question... >>> >>> Historically speaking, even though EVP_MAC and EVP_KDF are indeed new >>> APIs, they have a previous history of EVP APIs, through EVP_PKEY. The >>> impact of relocating them outside of the EVP "family" may be small, >>> but still, history gives me pause. >>> >>> RAND doesn't carry the same sort of history, which makes it much >>> easier for me to think "just do it and get it over with"... >> >>I have the same pause - so I'm thinking just RAND for now. >> >>Matt >> >> > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/
API renaming
There has been a suggestion to rename EVP_RAND to OSSL_RAND. This seems reasonable. Would it also make sense to rename the other new APIs similarly. More specifically, EVP_MAC and EVP_KDF to OSSL_MAC and OSSL_KDF respectively? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Reducing the security bits for MD5 and SHA1 in TLS
I’d agree it’s major for for SHA1 but not for MD5. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 18 Jun 2020, at 12:20 pm, Tim Hudson wrote: > > Given that this change impacts interoperability in a major way it should be a > policy vote of the OMC IMHO. > > Tim. > > > On Thu, 18 Jun 2020, 5:57 am Kurt Roeckx, <mailto:k...@roeckx.be>> wrote: > On Wed, May 27, 2020 at 12:14:13PM +0100, Matt Caswell wrote: > > PR 10787 proposed to reduce the number of security bits for MD5 and SHA1 > > in TLS (master branch only, i.e. OpenSSL 3.0): > > > > https://github.com/openssl/openssl/pull/10787 > > <https://github.com/openssl/openssl/pull/10787> > > > > This would have the impact of meaning that TLS < 1.2 would not be > > available in the default security level of 1. You would have to set the > > security level to 0. > > > > In my mind this feels like the right thing to do. The security bit > > calculations should reflect reality, and if that means that TLS < 1.2 no > > longer meets the policy for security level 1, then that is just the > > security level doing its job. However this *is* a significant breaking > > change and worthy of discussion. Since OpenSSL 3.0 is a major release it > > seems that now is the right time to make such changes. > > > > IMO it seems appropriate to have an OMC vote on this topic (or should it > > be OTC?). Possible wording: > > So should that be an OMC or OTC vote, or does it not need a vote? > > > Kurt >
Re: Backports to 1.1.1 and what is allowed
I’d also support 11968 being merged. I find the recent discussion and the likelihood of all major distros (supporting the S390x) picking up the patches fairly compelling, especially since the same people will be maintaining it. I would need to go and double check the non-S390x specific parts for possible breakage but the bulk of the changes are S390x specific. I support formalising the rules better than we have at the moment. Even if this is in conflict with the above. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 17 Jun 2020, at 1:43 pm, Benjamin Kaduk wrote: > > On Tue, Jun 16, 2020 at 02:03:58PM +0100, Matt Caswell wrote: >> PR 11188 proposes to backport a series of s390x patches to the 1.1.1 >> branch. IIUC it includes performance improvements as well as support for >> new hardware instructions. >> >> I think we need to have a much clearer and more explicit policy about >> exactly what is allowed to be backported to a stable branch. For example >> PR #11968 was a significant recent PR that backported AES CTR-DRBG >> performance improvements to the 1.1.1 branch and was allowed (should it >> have been?). > > I am happy to see 11968 landed; some informal testing that I hope to make > more formal soon seems to show a ~20% slowdown/regression for large RNG > requests going from 1.1.1d to 1.1.g, and 11968 provided substantially more > significant of a speedup (again, very informal testing, though). > > In this case you could say that the "bug" is that we're only calling AES on > a block at a time despite having much more effective backends available for > multi-block calls. > >> We have always said that the stable releases should only receive bug and >> security fixes. Performance improvements have been a bit of a grey area, >> e.g. a few lines of code that significantly improves the performance of >> a particular algorithm or codepath could reasonably be justified as >> "fixing a performance bug". OTOH bringing in whole new files of >> assembler seems to go way beyond that definition. > > There's probably some semi-subtle distinction to make along the lines of > "making the algorithm more efficient" vs "using a new algorithm, that > happens to run faster", with the latter counting as a feature. > >> However, when I tried to find a reference to the policy which says >> exactly what we are allowed to backport I could find one. Do we have >> such a thing? >> >> In any case I think we should develop a much more explicit policy that >> will enable us to decide whether PRs such as 11188 are reasonable or >> not. See for example Ubuntu's Stable Release Updates policy: > > I agree that having something written down will be very useful, even if > just as a fixed benchmark against which to think about how exceptional any > given "exception case" is where we want to deviate from it. > > -Ben > >> https://wiki.ubuntu.com/StableReleaseUpdates >> >> >> Matt
Seeking assistance
The OMC has approved working with the FIPS lab to include a number of additional algorithms in the upcoming FIPS validation. One slated for inclusion turns out to be beyond the time available for the fellows and the two others working on 3.0. This is the X9.42 KDF. What it needs is the X509 and CMS calls removed from the KDF and the various structures constructed piecemeal using calls that already exist within the FIPS provider. If somebody is willing to take this work on, it will likely be included in the FIPS validation for 3.0. If not, it won’t be. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Vote re: #11577
topic: Accept and merge #11577. comment: #11577 reduces the maximum length of TLS labels. It also breaks standards compliance. 8 against, none for, no abstentions, 3 not yet voted. The vote failed, the PR will be closed. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: OMC Vote on deprecation of command line apps
This vote has passed: 3 for, 1 against and 2 abstentions. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 8 May 2020, at 3:08 pm, Dr Paul Dale wrote: > > PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking > awaiting decision for a while now. Time for a vote: > > topic: Merge #11575 for 3.0. > comment: This PR removes the notes indicating that a number of the command > line utilities are deprecated. Not merging it will leave them > flagged > as deprecated. > Proposed by: Paul Dale > Public: yes > opened: 2020-05-08 > > Ideally we’ll have a decision in time for the next 3.0 alpha release. > > > The crux of the matter is that a number of the command line utilities are > flagged as deprecated currently: > dhparam > dsa > dsaparam > ec > ecparam > agendas > rsa > These commands are not being removed in 3.0, instead they’ve been rewritten > to use the PKEY APIs instead of the low level APIs as far as possible. > > > The reasons for keeping them are: > they are easier to use than the pkey replacements > a web search will likely result in thees commands not the pkey replacements. > > The reason for removing them is one of maintenance: having duplicate commands > means having to make changes in two places and this has been missed in the > past and will be in the future. > > > Other random notes: > Deprecation of these commands does not mandate that they are removed at the > first opportunity. It only indicates that we want to move away from them. > Rewriting these commands so that they call the pkey replacements looks to be > very difficult. Reproducing the exact behaviours will be challenging, > although the basic functionality would be straightforward. > The rsautl command is deprecated and isn’t slated for being restored — > pkeyutl is every bit as easy to use. > The -dsaparam option to dhparam is deprecated — it cannot be supported > without direct access to low level functionality we want to remove. > Post quantum crypto will make the discussion obsolete — none of these > algorithms are useful in a quantum computer world. > > My personal opinion is that these commands are good being deprecated but that > we should not remove them until their usefulness is at an end. This will > likely mean not removing them after five years of deprecation. It would mean > removing them once quantum computers are shown to be effective. Without > deprecation now, we can’t remove them until a lot later. > > > Pauli > -- > Dr Paul Dale | Distinguished Architect | Cryptographic Foundations > Phone +61 7 3031 7217 > Oracle Australia > > > >
OMC Vote on deprecation of command line apps
PR 11575 <https://github.com/openssl/openssl/pull/11575> has been blocking awaiting decision for a while now. Time for a vote: topic: Merge #11575 for 3.0. comment: This PR removes the notes indicating that a number of the command line utilities are deprecated. Not merging it will leave them flagged as deprecated. Proposed by: Paul Dale Public: yes opened: 2020-05-08 Ideally we’ll have a decision in time for the next 3.0 alpha release. The crux of the matter is that a number of the command line utilities are flagged as deprecated currently: dhparam dsa dsaparam ec ecparam agendas rsa These commands are not being removed in 3.0, instead they’ve been rewritten to use the PKEY APIs instead of the low level APIs as far as possible. The reasons for keeping them are: they are easier to use than the pkey replacements a web search will likely result in thees commands not the pkey replacements. The reason for removing them is one of maintenance: having duplicate commands means having to make changes in two places and this has been missed in the past and will be in the future. Other random notes: Deprecation of these commands does not mandate that they are removed at the first opportunity. It only indicates that we want to move away from them. Rewriting these commands so that they call the pkey replacements looks to be very difficult. Reproducing the exact behaviours will be challenging, although the basic functionality would be straightforward. The rsautl command is deprecated and isn’t slated for being restored — pkeyutl is every bit as easy to use. The -dsaparam option to dhparam is deprecated — it cannot be supported without direct access to low level functionality we want to remove. Post quantum crypto will make the discussion obsolete — none of these algorithms are useful in a quantum computer world. My personal opinion is that these commands are good being deprecated but that we should not remove them until their usefulness is at an end. This will likely mean not removing them after five years of deprecation. It would mean removing them once quantum computers are shown to be effective. Without deprecation now, we can’t remove them until a lot later. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
FIPS_mode() vote results
The vote: Remove the calls FIPS_mode() & FIPS_mode_set() in 3.0. Has closed. For: 3 Against: 1 Abstain: 2 The vote passes. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Cherry-pick proposal
My concern is are 1.1.1 and 3.0 really all that different? The core is quite different but the cryptographic algorithms aren’t. CMS, x509, …? I’d rather not introduce a burden where it isn’t necessary. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 29 Apr 2020, at 10:08 pm, Matt Caswell wrote: > > > The OTC have received this proposal and a request that we vote on it: > > I would like to request that we do not allow cherry-picks between master > and 1.1.1-stable because these two versions are now very different, if a > cherry-pick succeeds, there is no guarantee that the result will work. > Because we have no CI for the cherry-pick. If a cherry-pick is needed, a > new PR for 1.1.1 should be done and approved independently. > > Before starting a vote I'd like to provide opportunity for comments, and > also what the vote text should be. > > Thanks > > Matt
Re: An OpenSSL cookbook, where and how?
Might the demos be useful for something like this? I know they aren’t in great state and could do with better documentation but they seem to fulfil most of the suggested goals. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 7 Mar 2020, at 7:03 pm, Richard Levitte wrote: > > On Sat, 07 Mar 2020 10:01:59 +0100, > Richard Levitte wrote: >> >> As part of a documentation effort, more specifically in some >> discussions in >> https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220__;!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcJuDJ7pw$ >> (exact >> references below), I've floated the idea that bigger coding examples >> should be placed into a cookbook. > > I forgot the references: > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220*discussion_r386751635__;Iw!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5Bmc3tLgiYQ$ > > https://urldefense.com/v3/__https://github.com/openssl/openssl/pull/11220*issuecomment-596060267__;Iw!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcAjPjGdw$ > > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project > https://urldefense.com/v3/__http://www.openssl.org/*levitte/__;fg!!GqivPVa7Brio!IKN_xNCo3Vpfv_o5dvd-lE8BpdsZ0Pbcvjkm1_ee4kyvMZYTfI5w5BmcbK3Tc-Y$ >
Re: Deprecations
Matthew, Good idea. I’ll add it. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 5 Mar 2020, at 8:55 am, Matthew Lindner wrote: > > Shouldn't the deprecation notice that's printed also print the version it was > deprecated in? > > - Matthew Lindner > >> On Mar 4, 2020, at 2:48 PM, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote: >> >> EXTERNAL MAIL: openssl-project-boun...@openssl.org >> <mailto:openssl-project-boun...@openssl.org> >> >> Unless I’ve missed something, the documentation should specify the >> alternatives. Mostly these are one to one, but in one case it is one to two >> (and there both are listed). >> With the caveat that I might not have got every place or detail correct. >> >> Moreover, the deprecated commands print something to the effect of: "The >> command dsa is deprecated. Use ‘pkey’ instead." when executed. >> >> >> Pauli >> -- >> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >> >>> On 5 Mar 2020, at 5:15 am, Kurt Roeckx >> <mailto:k...@roeckx.be>> wrote: >>> >>> On Mon, Mar 02, 2020 at 11:41:57AM +, Matt Caswell wrote: >>>> >>>> >>>> On 28/02/2020 23:43, Dr Paul Dale wrote: >>>>> Any suggestions for a consensus on this thread? >>>> >>>> I think we can probably agree that: >>>> >>>> - Command option deprecations should be handled better >>>> - We should look at whether we can resurrect some of the "old" commands >>>> (possibly by writing them as wrappers for genpkey, pkey and pkeyutl) >>> >>> What about at least pointing to the alternative function in the >>> documentation? >>> >>> >>> Kurt >>> >> >
Re: Deprecations
Unless I’ve missed something, the documentation should specify the alternatives. Mostly these are one to one, but in one case it is one to two (and there both are listed). With the caveat that I might not have got every place or detail correct. Moreover, the deprecated commands print something to the effect of: "The command dsa is deprecated. Use ‘pkey’ instead." when executed. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 5 Mar 2020, at 5:15 am, Kurt Roeckx wrote: > > On Mon, Mar 02, 2020 at 11:41:57AM +, Matt Caswell wrote: >> >> >> On 28/02/2020 23:43, Dr Paul Dale wrote: >>> Any suggestions for a consensus on this thread? >> >> I think we can probably agree that: >> >> - Command option deprecations should be handled better >> - We should look at whether we can resurrect some of the "old" commands >> (possibly by writing them as wrappers for genpkey, pkey and pkeyutl) > > What about at least pointing to the alternative function in the > documentation? > > > Kurt >
Face to face
I propose that we don’t have either an OMC or OTC face to face meeting at ICMC. Travel restrictions and the availability list make it look unlikely. Instead, I’d suggest a video conference for both. This will probably require some kind of vote (for both). Any suggestions as to date and time. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Deprecations
I've started working on moving some of the old commands forward using PKEY calls. My intention is for them to still print out a deprecated message when run but for them to not actually be removed by the no-deprecated configure option. Having them print equivalent pkey command looks to be somewhat problematic. There isn’t a 1:1 conversion and some of the legacy options simply aren’t supported. I’m hoping to have a preliminary PR up later this week. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 2 Mar 2020, at 9:41 pm, Matt Caswell wrote: > > > > On 28/02/2020 23:43, Dr Paul Dale wrote: >> Any suggestions for a consensus on this thread? > > I think we can probably agree that: > > - Command option deprecations should be handled better > - We should look at whether we can resurrect some of the "old" commands > (possibly by writing them as wrappers for genpkey, pkey and pkeyutl) > > I am slightly concerned that the latter option (rewriting as wrappers) > may turn into a big black hole of effort. It *might* be easier to just > rewrite them as-is to use EVP. Whichever approach we take, I don't think > this should be a goal for alpha1. > > Matt > >> >> Pauli >> -- >> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >> >>> On 24 Feb 2020, at 5:08 pm, Dr Paul Dale >> <mailto:paul.d...@oracle.com>> wrote: >>> >>> Most of the conversions to using PKEY were straightforward. One >>> didn’t require any changes (dsa but my memory is suspect). One seemed >>> quite difficult. Some I didn’t check. >>> >>> Modifying the commands so that they continue to work and print (to >>> stderr) an alternative pkey based command might be workable too. >>> >>> >>> Pauli >>> -- >>> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >>> Phone +61 7 3031 7217 >>> Oracle Australia >>> >>> >>> >>> >>>> On 24 Feb 2020, at 5:53 am, Viktor Dukhovni >>>> mailto:openssl-us...@dukhovni.org>> wrote: >>>> >>>>> On Feb 22, 2020, at 4:53 AM, Richard Levitte >>>> <mailto:levi...@openssl.org>> wrote: >>>>> >>>>> Something that could be done is to take all those aged commands and >>>>> rewrite them as wrappers for genpkey, pkey and pkeyutl. Simply create >>>>> and populate a new argv and call genpkey_main(), pkey_main() or >>>>> pkeyutl_main(). >>>> >>>> Agreed, that sounds quite reasonable at first blush, and could be >>>> fantastic >>>> if it can be made to work (no immediate obstacles come to mind). >>>> >>>> -- >>>> Viktor. >>>> >>> >>
Re: Deprecations
Any suggestions for a consensus on this thread? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Feb 2020, at 5:08 pm, Dr Paul Dale wrote: > > Most of the conversions to using PKEY were straightforward. One didn’t > require any changes (dsa but my memory is suspect). One seemed quite > difficult. Some I didn’t check. > > Modifying the commands so that they continue to work and print (to stderr) an > alternative pkey based command might be workable too. > > > Pauli > -- > Dr Paul Dale | Distinguished Architect | Cryptographic Foundations > Phone +61 7 3031 7217 > Oracle Australia > > > > >> On 24 Feb 2020, at 5:53 am, Viktor Dukhovni > <mailto:openssl-us...@dukhovni.org>> wrote: >> >>> On Feb 22, 2020, at 4:53 AM, Richard Levitte >> <mailto:levi...@openssl.org>> wrote: >>> >>> Something that could be done is to take all those aged commands and >>> rewrite them as wrappers for genpkey, pkey and pkeyutl. Simply create >>> and populate a new argv and call genpkey_main(), pkey_main() or >>> pkeyutl_main(). >> >> Agreed, that sounds quite reasonable at first blush, and could be fantastic >> if it can be made to work (no immediate obstacles come to mind). >> >> -- >> Viktor. >> >
Re: Deprecations
Most of the conversions to using PKEY were straightforward. One didn’t require any changes (dsa but my memory is suspect). One seemed quite difficult. Some I didn’t check. Modifying the commands so that they continue to work and print (to stderr) an alternative pkey based command might be workable too. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Feb 2020, at 5:53 am, Viktor Dukhovni > wrote: > >> On Feb 22, 2020, at 4:53 AM, Richard Levitte wrote: >> >> Something that could be done is to take all those aged commands and >> rewrite them as wrappers for genpkey, pkey and pkeyutl. Simply create >> and populate a new argv and call genpkey_main(), pkey_main() or >> pkeyutl_main(). > > Agreed, that sounds quite reasonable at first blush, and could be fantastic > if it can be made to work (no immediate obstacles come to mind). > > -- > Viktor. >
Re: Deprecations
The added complexity was of some concern to me when doing the deprecations. I suspect we’ll also encounter difficulties getting 100% equivalent behaviour via PKEY. There are some pretty arcane options in some of these. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 22 Feb 2020, at 9:51 am, Kurt Roeckx wrote: > > On Fri, Feb 21, 2020 at 11:27:55PM +, Matt Caswell wrote: >> >> >> On 21/02/2020 23:18, Kurt Roeckx wrote: >>> On Fri, Feb 21, 2020 at 11:00:10PM +, Matt Caswell wrote: >>>> >>>> dhparam itself has been deprecated. For that reason we are not >>>> attempting to rewrite it to use non-deprecated APIs. The informed >>>> decision we have made about DH_check use in dhparam is to not build the >>>> whole application in a no-deprecated build: >>>> >>>> *) The command line utilities dhparam, dsa, gendsa and dsaparam have been >>>> deprecated. Instead use the pkeyparam, pkey, genpkey and pkeyparam >>>> programs respectively. >>>> [Paul Dale] >>> >>> For some reason I seem to have missed various things. >>> >>> But I think deprecating tools like dhparam, dsaparam in favour of >>> genpkey is something that we should reconsider. >> >> What is your reasoning? >> >> (I just realised that what the CHANGES entry says is that >> dhparam/dsaparam are deprecated in favour of pkeyparam - but actually I >> think the equivalent functionality is more split between genpkey and >> pkeyparam) > > Some equivalants: > openssl dhparam 2048 > openssl genpkey -genparam --algorithm DH -pkeyopt dh_paramgen_prime_len:2048 > > openssl dsaparam 2048 > openssl genpkey -genparam -algorithm DSA -pkeyopt dsa_paramgen_bits:2048 > > > If you search internet, you will more than likely find the first > ones. They are very easy. I have to look up at the manual page > examples to know how to use genpkey. > > > Kurt
Re: Errored: openssl/openssl#31939 (master - 34b1676)
An alternative would be to only run a cut down selection of tests with msan. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 14 Feb 2020, at 11:00 pm, Matt Caswell wrote: > > > > On 14/02/2020 12:23, Nicola Tuveri wrote: >> If ASAN is too slow to run in the CI should we restore the previous >> homemade checks for memory leaks as an alternative to be run in regular >> CI runs and leave ASAN builds to run-checker on the master branch only? > > To be clear the build that is timing out uses *msan* not *asan*. > > As I understand it msan detects unitialised reads. whereas asan detects > memory corruption, buffer overflows, use-after-free bugs, and memory leaks. > > The previous "home-made" checks only detected memory leaks, so it is not > comparable with the functionality offered by msan. > > The msan documentation > (https://urldefense.com/v3/__https://clang.llvm.org/docs/MemorySanitizer.html__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6xiEvnC0$ > ) suggests that a slow > down of 3x is typical. > > It seems reasonable to me to disable msan checks in Travis entirely, and > have them only in run-checker. > >> >> Here is another idea that would be interesting if we restore the >> previous checks: >> I don't know what kind of options github offers on this, but would it be >> possible to run triggered CI on something that is not Travis and does >> not timeout and still have the results in the PR? > > I am sure there are hooks to do this. Richard has been talking for quite > a while about setting up a buildbot infrastructure. If that could be > integrated into github that would be really neat. > > Matt > > >> If something like that would be possible we could move the ASAN builds >> to extended_tests, rely on the previous memleak detection for the >> regular CI runs, and then trigger with a script or Github Action the >> extended_tests when the approval:done label is added. >> >> That way, by the time something is ready to be merged we should have a >> full picture! >> >> >> Nicola >> >> On Wed, Feb 5, 2020, 10:25 Matt Caswell > <mailto:m...@openssl.org>> wrote: >> >>Since we fixed the Travis builds 4 out of the 8 builds on master that >>have taken place have errored due to a timeout. >> >>The msan build is consistently taking a *very* long time to run. If it >>gets to 50 minutes then Travis cuts it off and the build fails. >> >>Should we disable the msan build? >> >>Matt >> >> >> Forwarded Message >>Subject:Errored: openssl/openssl#31939 (master - 34b1676) >>Date: Wed, 05 Feb 2020 00:02:01 + >>From: Travis CI mailto:bui...@travis-ci.org>> >>To: openssl-comm...@openssl.org <mailto:openssl-comm...@openssl.org> >> >> >> >>openssl >> >>/ >> >>openssl >> >> >> <https://urldefense.com/v3/__https://travis-ci.org/openssl/openssl?utm_medium=notification&utm_source=email__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6bPlKFFo$ >> > >> >> >>branch iconmaster >> <https://urldefense.com/v3/__https://github.com/openssl/openssl/tree/master__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6tYedpt0$ >> > >> >>build has errored >>Build #31939 has errored >> >> <https://urldefense.com/v3/__https://travis-ci.org/openssl/openssl/builds/646181069?utm_medium=notification&utm_source=email__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6dntSzqk$ >> > >>arrow to build time >>clock icon50 mins and 3 secs >> >>Pauli avatarPauli >> >>34b1676 CHANGESET → >> >> <https://urldefense.com/v3/__https://github.com/openssl/openssl/compare/e3b1ccad694a...34b167625af5__;!!GqivPVa7Brio!JwD52xjNRP5yVXTD3K12Mn17HiC2xHM_O6YzFE7G32G1BYh-NID9mIM6KOlK4nk$ >> > >> >>Make minimum size for secure memory a size_t. >> >>The minimum size argument to CRYPTO_secure_malloc_init() was an int but >>ought >>to be a size_t since it is a size. >> >>From an API perspective, this is a change. However, the minimum size is >>verified as being a positive power of two and it will typically be a >>small >>constant. >> >>Reviewed-by: David von
Re: Deprecation
Roumen in https://github.com/openssl/openssl/pull/10977#issuecomment-584818517 <https://github.com/openssl/openssl/pull/10977#issuecomment-584818517> Dmitry in https://github.com/openssl/openssl/pull/11082#issuecomment-585603911 <https://github.com/openssl/openssl/pull/11082#issuecomment-585603911> And a further one via private email. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 14 Feb 2020, at 7:37 pm, Matt Caswell wrote: > > > > On 14/02/2020 02:30, Dr Paul Dale wrote: >> There is some pushback against the deprecations going on in various PRs. > > I've not followed all of the PRs. Can you point at some specific > comments you've received pushing back on this? > > Matt > > >> >> The plan has always been to deprecate engines in 3.0 and removing >> support for them 5+ years later. Originally, the path was to have >> included an engine provider that could load engines and make them appear >> to be a provider. After a fair amount of investigation, this was deemed >> to be too difficult in the 3.0 time frame. >> >> Do we still want to deprecate engines in 3.0? >> Should we defer until 4.0 instead? >> >> >> The main benefits seem to boil down to continuing to support existing >> engines vs removing the legacy code paths and switching to the provider >> model. >> >> >> Pauli >> -- >> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >> Phone +61 7 3031 7217 >> Oracle Australia >>
Deprecation
There is some pushback against the deprecations going on in various PRs. The plan has always been to deprecate engines in 3.0 and removing support for them 5+ years later. Originally, the path was to have included an engine provider that could load engines and make them appear to be a provider. After a fair amount of investigation, this was deemed to be too difficult in the 3.0 time frame. Do we still want to deprecate engines in 3.0? Should we defer until 4.0 instead? The main benefits seem to boil down to continuing to support existing engines vs removing the legacy code paths and switching to the provider model. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Github PR label automation
Please don’t automatically drop the "appoval: done" label after a comment. I feel that is not uncommon for comments to be added that in no way invalidate the approval. I agree with not switching to “ready to merge” if there are comments — manual intervention in this case is required to judge the relevancy. Agreed also over the “urgent” label. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 9 Feb 2020, at 1:56 am, Mark J Cox wrote: > > I've currently got a cron job running every hour that looks at open PR > requests against github openssl repo and does various actions. So if > you were wondering why I was altering labels and making comments at > 4am, now you know. No doubt we'll use some tool user for this in the > future. > > So right now here's what it does: > > Every hour it looks at open PRs that are labelled "approval: done". > If 24 hours has elapsed since that label was assigned and if there > have been no comments made to the PR since the label was assigned then > it is automatically moved to "approval: ready to merge" with a comment > added to trigger notifications. So if you want to stop something > going to "ready to merge" just add any comment to the PR. > > I'm thinking of using this script also to 1) collect interesting stats > and 2) do some other actions. So if there's some automation you'd > like to see just add an enhancement issue against the openssl/tools > repo. > > 1 Matt already asked for committer notification trigger for anything > labelled Urgent. > > 2 If there were comments made after "approval: done" then I think we > really ought to drop the "approval: done" label as the comments likely > invalidated the approval. So I'll likely add that next week (if > "approval: done" label and has comments since that label then remove > the label and add a comment 'please review if this is really approval: > done'. If the approval: done label gets set again then after 24 hours > the existing automation will trigger. #10786 is a good example of > this. > > Mark
Re: Travis in solid red mode again
I thought I was subscribed but don’t seem to see the failures. I do get the (very many) PR activity emails…. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 1 Feb 2020, at 8:35 pm, Dr. Matthias St. Pierre > wrote: > >> -Original Message- >> On 01/02/2020 00:34, Salz, Rich wrote: >>> >>>> Could we add a CI check for PRs that confirms that the target branch is >>>green in travis? >>> >>> Looking at >>> https://docs.travis-ci.com/user/notifications/#conditional-notifications >>> and https://config.travis- >> ci.com/ref/notifications/email it seems like it should be fairly easy >> configure builds to do "send email to openssl-commits when builds on >> master fail" >> >> We already have that. > > Maybe we should make it a requirement that at least all OTC members subscribe > to openssl-commits? > > Matthias >
Re: crypt(3)
Thanks for the feedback everyone. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: crypt(3)
I meant “what default makes the most sense for the passwd command line application?” It was crypt which is deprecated. Should it be BSD’s MD5? One of the SHA2 based algorithms? Or should it produce an error if no algorithm is selected? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 19 Jan 2020, at 12:28 am, Kurt Roeckx wrote: > > On Sat, Jan 18, 2020 at 10:47:04AM +1000, Dr Paul Dale wrote: >> Could the people who work with distros confirm this default choice or >> suggest what they use please? > > I'm not sure what you're asking, but crypt() has moved on from > using DES like 20 years ago, see crypt(5). > > > Kurt >
Re: crypt(3)
Could the people who work with distros confirm this default choice or suggest what they use please? Thanks, Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 18 Jan 2020, at 10:05 am, Dr Paul Dale wrote: > > I’ve made the deprecation changes to the password application. > > The default has been changed from crypt to the BSD MD5 algorithm. > > Pauli > -- > Dr Paul Dale | Distinguished Architect | Cryptographic Foundations > Phone +61 7 3031 7217 > Oracle Australia > > > > >> On 18 Jan 2020, at 9:27 am, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote: >> >> Okay, it looks like the consensus is option 3 — deprecate and forget. >> >> As far as I can tell, they are only used (by us) in one place outside of >> libcrypto, so that will deprecate as well. >> >> >> Pauli >> -- >> Dr Paul Dale | Distinguished Architect | Cryptographic Foundations >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >> >>> On 18 Jan 2020, at 6:53 am, Richard Levitte >> <mailto:levi...@openssl.org>> wrote: >>> >>> Right. Such a KDF could be implemented elsewhere, as a separate project. >>> >>> Cheers >>> Richard >>> >>> >>> Kurt Roeckx mailto:k...@roeckx.be>> skrev: (17 januari >>> 2020 21:35:00 CET) >>>> On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote: >>>>> I’ve got several choices: >>>>> Leave them public and unchanged — that is, don’t deprecate these two >>>> functions yet. >>>>> Deprecate them and add KDFs to replace them. >>>>> Deprecate them, leave them alone and hope they go away painlessly at >>>> some point. >>>> >>>> I really see no point in adding something that we at the same time >>>> would like to remove. Just deprecate it. >>>> >>>> >>>> Kurt >>> >>> -- >>> Richard by mobile >> >
Re: crypt(3)
Okay, it looks like the consensus is option 3 — deprecate and forget. As far as I can tell, they are only used (by us) in one place outside of libcrypto, so that will deprecate as well. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 18 Jan 2020, at 6:53 am, Richard Levitte wrote: > > Right. Such a KDF could be implemented elsewhere, as a separate project. > > Cheers > Richard > > > Kurt Roeckx skrev: (17 januari 2020 21:35:00 CET) >> On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote: >>> I’ve got several choices: >>> Leave them public and unchanged — that is, don’t deprecate these two >> functions yet. >>> Deprecate them and add KDFs to replace them. >>> Deprecate them, leave them alone and hope they go away painlessly at >> some point. >> >> I really see no point in adding something that we at the same time >> would like to remove. Just deprecate it. >> >> >> Kurt > > -- > Richard by mobile
Re: crypt(3)
My experience with embedded systems is that crypt(3) is in the standard library and not accessed via OpenSSL. Apps/password uses DES_crypt() and password crackers used to use OpenSSL for performance reasons. Neither seems like a huge deal. I.e. I can’t think of a good reason to keep them. Removing these calls will require an OMC vote as a breaking API change. I’m fine to call one if it seems justified. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 17 Jan 2020, at 5:41 pm, Viktor Dukhovni > wrote: > > On Fri, Jan 17, 2020 at 04:31:06PM +1000, Dr Paul Dale wrote: > >> There are two functions (DES_crypt and DES_fcrypt) which implement the >> old crypt(3) password algorithm. Once these are deprecated, they will >> no longer be reachable via EVP. The confounding point is that they >> aren’t quite DES — close but not identical. I would be surprised if >> they aren’t still in use for /etc/passwd files on old and/or embedded >> systems. > > Generally speaking, on Unix-like systems that use crypt(3) for > /etc/passwd I'd expect to find a standaline crypt() implementation in > libc, that is independent of OpenSSL. That is, if your system still > uses crypt() for passwords, you don't need OpenSSL to compute crypt > hashes. > > That said, this is experience from general-purpose computers running > Unix-like OSes, not embedded systems, where I have no idea whether > crypt() is popular, and whether it is provided by a port of libcrypto > to that platform. > >> I’ve got several choices: Leave them public and unchanged — that is, >> don’t deprecate these two functions yet. Deprecate them and add KDFs >> to replace them. Deprecate them, leave them alone and hope they go >> away painlessly at some point. > > I would not expect to find many users of OpenSSL's crypt(), except > internally within OpenSSL itself. > >> The apps/password.c applet calls these which is how I stumbled over >> the complication. I’m fine refactoring this based on the solution >> chosen. I’d also be okay with factoring out all the password >> derivation functions into KDFs if necessary. >> >> Thoughts? Other alternatives? > > I don't know enough about embedded systems to speak about what if > anything we need to do for those with respect to crypt(). > > -- >Viktor.
crypt(3)
In the deprecation efforts for 3.0, I’ve hit something in the DES code that I’d appreciate input on. There are two functions (DES_crypt and DES_fcrypt) which implement the old crypt(3) password algorithm. Once these are deprecated, they will no longer be reachable via EVP. The confounding point is that they aren’t quite DES — close but not identical. I would be surprised if they aren’t still in use for /etc/passwd files on old and/or embedded systems. I’ve got several choices: Leave them public and unchanged — that is, don’t deprecate these two functions yet. Deprecate them and add KDFs to replace them. Deprecate them, leave them alone and hope they go away painlessly at some point. The apps/password.c applet calls these which is how I stumbled over the complication. I’m fine refactoring this based on the solution chosen. I’d also be okay with factoring out all the password derivation functions into KDFs if necessary. Thoughts? Other alternatives? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Legacy provider
I’m not sure what more I can write. I proposed the vote text around the time I sent the notification here: no comments. I created the vote, early in the voting period, the clarification was sought and made. All OMC members registered their vote and the vote closed early. The criteria for being valid as per the bylaws <https://www.openssl.org/policies/omc-bylaws.html> was met. As votes go, this one was quick taking two days of the two weeks. Abstentions are frequent in votes for a number of reasons. The reasons each person uses are not revealed and not asked for. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 16 Jan 2020, at 6:07 am, Benjamin Kaduk wrote: > > Hi Pauli, > > On Tue, Jan 14, 2020 at 09:34:40PM +1000, Dr Paul Dale wrote: >> The OMC vote is closed. >> >> The vote text being: >> >> The legacy provider should be disabled by default in 3.0 >> >> With the clarification that "disabled" in this context means "not loaded”. >> >> The vote passed (two for, one against, four abstain) > > It's good to have a decision here, but I'm kind of worried about the four > abstains -- it's easy for me to leap to a conclusion that the individuals > in question just didn't want to to spend the time to come to a considered > position, even though this issue has substantial potential impact for our > userbase. I'm trying to not make faulty assumptions, so some greater > clarity on the circumstances would be helpful, if possible. > > Thanks, > > Ben
Legacy provider
The OMC vote is closed. The vote text being: The legacy provider should be disabled by default in 3.0 With the clarification that "disabled" in this context means "not loaded”. The vote passed (two for, one against, four abstain) Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Legacy Provider
Kurt, It’s a policy decision: should we cause pain for users (& Matt) or effectively delay the end for these old/broken algorithms. Technically it is easy. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 9 Jan 2020, at 6:30 am, Kurt Roeckx wrote: > > On Tue, Jan 07, 2020 at 09:57:55AM +1000, Dr Paul Dale wrote: >> The refactoring/FIPS work needs the question resolved about loading the >> legacy provider or not by default. We’ve been through this before on the >> project list [1] and in at least one PR [2]. >> >> I expect that its resolution will require an OMC vote. > > Why OMC and not OTC? > > > > Kurt >
Legacy Provider
The refactoring/FIPS work needs the question resolved about loading the legacy provider or not by default. We’ve been through this before on the project list [1] and in at least one PR [2]. I expect that its resolution will require an OMC vote. Regardless of the outcome, the current intention is that the low level direct access functions (e.g. IDEA_encrypt) will continue to work (albeit deprecated), only the EVP access will go (again, by default). Before the vote is called, are there any additional thoughts from the past six months? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia [1] https://mta.openssl.org/pipermail/openssl-project/2019-July/001492.html [2] https://github.com/openssl/openssl/pull/9373
Re: Flaw in our process for dealing with trivial changes
A better example of this problem: #10607. Both Paul and I approved it yesterday and I merged it today without noticing until too late that it was tagged “CLA: trivial” :( I’ve not reverted it at this point but will if necessary. Let’s get the label in. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 13 Dec 2019, at 11:02 am, Richard Levitte wrote: > > On Thu, 12 Dec 2019 22:31:10 +0100, > Dr Paul Dale wrote: >> >> A red blocker along the lines of: “Triviality Unconfirmed”. One of >> the reviewers needs to remove this before the PR can be merged. >> >> It’s in our face, it prevent accidental merges and its low overhead. > > I still think simply adding the label should be sufficient. I dunno > about you, but I look at labels all the time, for all sorts of > reasons, and one saying [cla: trivial] would certainly attract my > attention. > > Let's make it bright red-orange, that'll catch anyone's eye (even mine) > > Also, removing that label will rapidly be annoying as soon as someone > closes and re-opens a PR... or whatever other action that triggers > the "pull_request" event (and there's a lot that does that... our > script is being kept busy!). > > Cheers, > Richard > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/
Re: Flaw in our process for dealing with trivial changes
A red blocker along the lines of: “Triviality Unconfirmed”. One of the reviewers needs to remove this before the PR can be merged. It’s in our face, it prevent accidental merges and its low overhead. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 13 Dec 2019, at 7:06 am, Dr Paul Dale wrote: > > Before we start over engineering a solution, how about we try just having an > automatic visual indicator for trivial PRs. > > > Pauli > -- > Dr Paul Dale | Distinguished Architect | Cryptographic Foundations > Phone +61 7 3031 7217 > Oracle Australia > > > > >> On 13 Dec 2019, at 3:24 am, Kurt Roeckx > <mailto:k...@roeckx.be>> wrote: >> >> On Thu, Dec 12, 2019 at 12:10:35PM +, Matt Caswell wrote: >>> >>> But in principle I agree that addrev could be used to do this. It's not >>> quite as robust as doing it in the commit hook - because you don't >>> *have* to use addrev. But, AFAIK, everyone does - so that's probably >>> good enough. >> >> I have never used addrev. >> >> >> Kurt >> >
Re: Flaw in our process for dealing with trivial changes
Before we start over engineering a solution, how about we try just having an automatic visual indicator for trivial PRs. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 13 Dec 2019, at 3:24 am, Kurt Roeckx wrote: > > On Thu, Dec 12, 2019 at 12:10:35PM +, Matt Caswell wrote: >> >> But in principle I agree that addrev could be used to do this. It's not >> quite as robust as doing it in the commit hook - because you don't >> *have* to use addrev. But, AFAIK, everyone does - so that's probably >> good enough. > > I have never used addrev. > > > Kurt >
Re: Flaw in our process for dealing with trivial changes
I agree that there is a possible flaw in the workflow. What’s saved us so far is that new contributors don’t generally include the "CLA: trivial" line or put it in the GitHub text. Could we have a “trivial” tag that is added whenever the "CLA: trivial" line is present? Better would be to add it only if the submitter doesn’t have a CLA on file but either works. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 12 Dec 2019, at 7:20 pm, Matt Caswell wrote: > > I notice that PR 10594 (Add support for otherName:NAIRealm in output) > got merged yesterday: > https://github.com/openssl/openssl/pull/10594 > > The commit description contained "CLA: trivial" and so the "hold: cla > required" label was not automatically applied to the PR. But the > discussion in the PR suggested a CLA should be submitted. But it got > merged anyway! Fortunately the CLA had already been processed - just not > noted in the PR. So, in this case, it makes no difference. > > I think this points to a possible flaw in our workflow for dealing with > trivial changes. Because the "CLA: trivial" header suppresses the "hold: > cla required" label and the git hooks don't complain when commits get > pushed with the "CLA: trivial" header and no CLA on file, it seems > possible to me that we could push commit all the way through the process > without the reviewers even realising that the author is claiming > triviality on the commit. > > Not sure what the solution to that is. > > Matt
Re: Check NULL pointers or not...
Oops, you are correct. I was under the mistaken impression that ossl_assert
compiled to nothing outside of debug mode.
Pauli
--
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations
Phone +61 7 3031 7217
Oracle Australia
> On 29 Nov 2019, at 7:22 pm, Matt Caswell wrote:
>
>
>
> On 29/11/2019 08:38, Dr Paul Dale wrote:
>> I’d prefer option 1 or the middle ground. I’ve lost count of the
>> number of times I’ve seen programs crashing in the crypto library
>> which required mammoth debugging efforts to irrefutably demonstrate
>> that the caller is at fault rather than the crypto library :(
>>
>> Option 1 would be preferable from this point of view but it can cause
>> a performance hit — most of the time it wouldn’t matter but when it
>> does it would be a big deal. The middle ground doesn’t entail any
>> performance loss in production code (it does in debug but that
>> shouldn’t be relevant).
>
>
> I think you misunderstand the middle ground option:
>
>if (!ossl_assert(ptr != NULL)) {
>ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER);
>return 0;
>}
>
> In debug code this will crash if ptr is NULL. In production code this
> acts exactly like option 1 - so has exactly the same performance hit.
>
> For the record my preference is the middle ground option as being the
> norm for new code and where we make a significant refactor of old code.
> If something truly is performance critical then we can choose not to do
> it in those cases.
>
> Matt
>
Re: Check NULL pointers or not...
I’d prefer option 1 or the middle ground. I’ve lost count of the number of
times I’ve seen programs crashing in the crypto library which required mammoth
debugging efforts to irrefutably demonstrate that the caller is at fault rather
than the crypto library :(
Option 1 would be preferable from this point of view but it can cause a
performance hit — most of the time it wouldn’t matter but when it does it would
be a big deal. The middle ground doesn’t entail any performance loss in
production code (it does in debug but that shouldn’t be relevant).
Pauli
--
Dr Paul Dale | Distinguished Architect | Cryptographic Foundations
Phone +61 7 3031 7217
Oracle Australia
> On Wed, Nov 27, 2019 at 10:38:41AM +0100, Richard Levitte wrote:
>> Depending on who you're asking, you get different responses.
>>
>> The topic is: should we check pointers coming from applications before
>> using them or not?
>>
>> There are two positions on this:
>>
>> 1. Yes we should so we don't crash processes unnecessarily, and we
>> should return an error if pointers are NULL (unless that's valid
>> input, such as for the freeing functions). There is a generic
>> error reason code for it, ERR_R_PASSED_NULL_PARAMETER.
>>
>> 2. No we should not, as we don't cater for user programming errors.
>> Also, it allows us to catch our own errors early.
>>
>> For a while, our pendulum was going with option 2, but lately, we seem
>> to go back to option 1.
>>
>> Both options have their pros and cons:
>>
>> 1. cons: there's the very real possibility that users don't check for
>> errors as they should, and things go wrong further on,
>> sometimes in ways where the actual cause is very hard to
>> figure out.
>> pros: programs will not spuriously crash because we didn't catch an
>> internal corner case in our tests.
>>
>> 2. cons: programs may crash, sometimes due to our own mistakes,
>> sometimes due to user errors.
>> pros: some very subtle bugs are found, either quicker or at all.
>> An actual case is PR 7630 [1], which was uncovered because
>> there was a crash due to a NULL being used (this was
>> ultimately due to users not checking errors).
>>
>> As a middle ground, and perhaps a way to satify both camps, we could
>> use ossl_assert() in our checks for input NULL, as follows:
>>
>>if (!ossl_assert(ptr != NULL)) {
>>ERR_raise(ERR_LIB_WHATEVER, ERR_R_PASSED_NULL_PARAMETER);
>>return 0;
>>}
>>
>> This means that in a debug build, there may be crashes due to pointers
>> being NULL, while in a release build, the caller gets an error.
>>
>> Thoughts?
Re: Malloc failures check
Adding a compile time check would be good. I’m not sure how. It would be possible to implement a malloc failure feature in the test suite that systematically runs a test many times, failing successive malloc calls. I’m kind of surprised that the various static analysers hadn't found some of these. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 21 Nov 2019, at 1:26 pm, Dmitry Belyavsky wrote: > > Hello, > > Observing a series of similar bugs related to a lack of checks of the malloc > return values, I wonder if we could automate the search of these errors on > the compile level (e.g. similar to the __owur macro)? > > -- > SY, Dmitry Belyavsky
Re: #10388
The consensus seems to be to add the deprecated API to 3.0. I’ve removed the hold. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 15 Nov 2019, at 10:40 pm, Matthias St. Pierre > wrote: > > > > On 14.11.19 20:40, Viktor Dukhovni wrote: >>> On Nov 14, 2019, at 9:15 AM, Matt Caswell wrote: >>> >>> "No existing public interface can be removed until its replacement has >>> been in place in an LTS stable release. The original interface must also >>> have been documented as deprecated for at least 5 years. A public >>> interface is any function, structure or macro declared in a public >>> header file." >>> >>> So the functions we're talking about don't yet have a replacement - >>> there will be one in 3.0. So presumably they will be documented as >>> deprecated in 3.0. So we have to support them for at least 5 years from >>> the release of 3.0. >> I think that when we're deprecating an entire family of *related* accessors >> (for the same structure) that have been in place for some time, the addition >> of a missing member of that family is reasonably interpreted as not resetting >> the support clock on the entire family. We can still remove them all as >> though >> the missing members of the family had been in place all along. >> >> That is, we should (and if that requires a policy update and vote so be it) >> be >> able to interpret the rules based on the "spirit" (intent) without getting >> unduly burdened by the "letter", where common sense would suggest that we're >> getting the intended outcome. > > I don't think we need a reinterpretation, or even a change, of the existing > policy for > this specific case, since already now the *entire* engine api can only be > deprecated > in version 3.0 and removed afterwards according to the policy cited by Matt. > > We can simply add the missing accessor as bugfix to 1.1.1 and 3.0, and > immediately > deprecate it on the latter. The only difference will be that we end up with > n+1 deprecated > functions instead of n (for some natural number n) for version 3.0. > > Even after 3.0 is released and as long as 1.1.1 LTS is still supported, we > can proceed > in the same way without violating existing policies. > > > Matthias
#10388
With reference to #10388: https://github.com/openssl/openssl/pull/10388 <https://github.com/openssl/openssl/pull/10388> This PR intends to add missing APIs to 1.1.1. I’m fine with this being considered a bug and adding them. I’m more concerned about adding these to 3.0. It means we’ll have to support the new API for a long time and it is one which we are currently trying to move away from. Thoughts or comments anyone? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Commit access to openssl/tools and openssl/web
I believed that it required two OMC approvals but was pointed to an earlier instance where only one was present and I flew with it without checking further. My apologies for merging prematurely and I’ll back out the changes if any OMC member wants. As for discussing this at the upcoming face to face, I agree wholeheartedly. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 4 Oct 2019, at 5:39 pm, Matt Caswell wrote: > > > > On 04/10/2019 08:15, Dr. Matthias St. Pierre wrote: >> Dear OMC, >> >> while the process of merging and committing to openssl/openssl has been >> formalized, >> no similar (official) rules for pull requests by non-OMC-member seem to >> apply to the >> other two repositories openssl/tools and openssl/web. Probably it's because >> hardly >> anybody outside the OMC else ever raises them? Or is it the other way around? > > There are clear official rules. This vote was passed by the OMC over a year > ago: > > topic: Openssl-web and tools repositories shall be under the same review > policy as per the openssl repository where the reviewers are OMC members > > So it needs two approvals from an OMC member. It looks like recent commits > haven't obeyed those rules. > > >> I would like to raise the question whether it wouldn't be beneficial for all >> of us, >> if we would apply the same rules (commit access for all committers, plus the >> well >> known approval rules) to all of our repos. After all, the openssl/openssl >> repository >> is the most valuable of the three and I see no reason why the others would >> need >> more protection. In the case of the openssl/web repository which targets the >> official website, you might want to consider a 2OMC approval rule, but even >> there >> I don't see why the usual OMC veto rule wouldn't be sufficient. > > There is a lot of merit in that. Certainly for tools. I've added it to the OMC > agenda for Nuremburg. > > Matt >
Re: Reorganization of the header files (GitHub #9333)
Go for it, the antipodean contingent aren’t busy this weekend. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 28 Sep 2019, at 5:05 pm, Dr. Matthias St. Pierre > wrote: > >> Merge early is pretty much my default position ... and that applies to this >> context in my view. > > Thank you Tim for your quick and clear consent. > > Meanwhile, pull request #9333 and its counterpart #9681 for the 1.1.1 stable > branch have been approved by Richard. Unless I hear any protest until Sunday > afternoon (CEST), I would like to merge in the evening, just in time before > the > Australian plumbers start their weekly shift again. > > Matthias
Re: RAND, FIPS and providers
Matt, thanks for the clarification. I’ve looked at the DRBG setup code dozens of times and it never clicked. It seems we’re down to making the DRBGs and, perhaps, the seed source available using fetch. That doesn’t seem anything like as difficult. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 24 Sep 2019, at 6:32 pm, Matt Caswell wrote: > > > > On 24/09/2019 01:17, Dr Paul Dale wrote: >> This started in #9954, the topic of RAND being used by the legacy provider >> came up (in the context of DES). The abridged version is: >> >> * @levitte suggested the possibility of making RAND detachable. >> * I noted that this was desirable and in fact necessary for FIPS. >> * @mattcaswell added that the DRBGs and seeding is available inside the FIPS >> provider. >> >> >> That the FIPS provider includes a copy of the relevant RAND files, means it >> can satisfy internal requests for random numbers. >> However, external entities (TLS stack, user applications) won’t git FIPS >> approved random numbers. >> >> I can’t see currently an alternative to making the RAND functionality >> fetchable. > > I think making RAND fetchable is highly desirable and should be done (I had > always assumed we would do this). > >> I also suspect it will need to be per library context which might interfere >> with the per thread DRBGs we’re using. > > I see no problems here. The RAND code is already library context aware. You > get > per thread DRBGs for each OPENSSL_CTX. For example calling > OPENSSL_CTX_get0_private_drbg() will get you the private DRBG for the current > thread in the specified OPENSSL_CTX. RAND_DRBG_get0_private() does the same > thing for the default OPENSSL_CTX. > > >> As for what to fetch: the DRBG instances and the seed material source would >> be ideal, although we don’t need the seed source for FIPS (so long as the >> DRBGs seed from inside their own provider). > > I had always assumed we would fetch DRBG instances. > > Matt > >> >> >> Thoughts or input anyone? >> >> >> Pauli >>
RAND, FIPS and providers
This started in #9954, the topic of RAND being used by the legacy provider came up (in the context of DES). The abridged version is: * @levitte suggested the possibility of making RAND detachable. * I noted that this was desirable and in fact necessary for FIPS. * @mattcaswell added that the DRBGs and seeding is available inside the FIPS provider. That the FIPS provider includes a copy of the relevant RAND files, means it can satisfy internal requests for random numbers. However, external entities (TLS stack, user applications) won’t git FIPS approved random numbers. I can’t see currently an alternative to making the RAND functionality fetchable. I also suspect it will need to be per library context which might interfere with the per thread DRBGs we’re using. As for what to fetch: the DRBG instances and the seed material source would be ideal, although we don’t need the seed source for FIPS (so long as the DRBGs seed from inside their own provider). Thoughts or input anyone? Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia
Re: Being socially aware
I’m not disputing the great effort put into this. My dispute is that it should be under the openssl list command….. I agree, this shouldn’t have been a “good first issue”. Pauli -- Dr Paul Dale | Distinguished Architect | Cryptographic Foundations Phone +61 7 3031 7217 Oracle Australia > On 17 Sep 2019, at 3:49 pm, Richard Levitte wrote: > > ... or how our usual style of dispute can sometimes deter help from > the community. > > https://github.com/openssl/openssl/pull/9912 > > In this PR, there's a lot of discussions going on, a bit back an > forth, about the right way to do things, and what does what like what > other thing and so on. > > This is often our modus operandi, and it has often given us pretty > good quality code (or at least, so we hope), even though it can be a > bit exhausting at times. > > I would like to emphasise that I think this is absolutely fine... > when it's among us who have been along for a bit, perhaps have come to > know each other a bit more and have some kind of sense of our > respective strengths and weaknesses, or even with someone that has > shown they can handle this type of discussion. > > However, in pull requests like the one cited, where the associated > issue is marked "good first issue", and the author has done quite well > what was asked from the issue, it can be quite unexpected, not to say > overwhelming to be met with this discussion style that could be seen > as getting out of proportion. A "good first issue" is supposed to be > a bite size thing after all, and I fear that if this is how we welcome > new contributors, they might more often than not choose to step away > from it all. > > So maybe let's be a little more careful with contributions for "good > first issue" and potential newcomers, yeah? > > Cheers, > Richard > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/
Re: Thread sanitiser problems
The locks aren’t being held for long. I have noticed that the lock in the OPENSSL_CTX only seems to be locked for reading and only in one place. crypto/context.c in openssl_ctx_get_data(). Writes seem to go via CRYPTO_alloc_ex_data() which uses a global write lock (on a different lock). Would it be feasible to drop the OPENSSL_CTX lock completely? This would address the issue. Matt? I think you know this code the best. CRYPTO_alloc_ex_data() is calling the new_func pointer under the read lock and the new function can in turn cause a search of the providers for implementations which grabs the provider lock. Conversely, searching the providers for implementations grabs the provider lock and can lead to openssl_ctx_get_data() grabbing the context lock. We don’t appear to have recursion at the moment, but the setup seems a little fragile. We do have two different ordering for grabbing locks which is also bad. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 31 Jul 2019, at 2:10 pm, Viktor Dukhovni > wrote: > >> On Jul 30, 2019, at 10:02 PM, Dr Paul Dale wrote: >> >> The #9454 description includes thread sanitisizer logs showing different >> lock orderings — this has the potential to dead lock. Agreed with Rich that >> giving up the lock would make sense, but I don’t see a way for this to be >> easily done. > > My take is that we should never hold any lock long enough to even consider > acquiring another lock. No more than one lock should be held at any one > time, and only long enough to bump a reference count to ensure that the > object of interest is no deallocated before we (or our caller in a "get1" > type interface) is done with it. > > I don't know what "long-term" locks we're holding, but it would be great > if it were possible to never (or never recursively) hold any such locks. > > -- > Viktor. >
Re: Thread sanitiser problems
Yes, I’m mostly talking about #9454 here. #9455 is a bug (clearing the flush flag after flushing not before). The fix in #9477 addresses this and also removes the dependence on RAND_bytes. The #9454 description includes thread sanitisizer logs showing different lock orderings — this has the potential to dead lock. Agreed with Rich that giving up the lock would make sense, but I don’t see a way for this to be easily done. That this also involves the RAND subsystem could be coincidental. The other stack trace is getting an MD via a digest operation (i.e. both traces are for algorithms that use other algorithms). The locks in question are the provider store lock and the context lock. crypto/context.c: OPENSSL_CTX *ctx; /* function argument */ CRYPTO_THREAD_read_lock(ctx->oncelock); crypto/provider_core.c: OPENSSL_CTX *ctx; /* function argument */ struct provider_store_st *store = get_provider_store(ctx); CRYPTO_THREAD_read_lock(store->lock); Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 30 Jul 2019, at 8:52 pm, Matthias St. Pierre > wrote: > > Sorry, my reply was misleading, since Pauli is talking mainly about #9454. > Please take a look at the issue description > > https://github.com/openssl/openssl/issues/9454 > > instead. > > Matthias > > > > On 30.07.19 12:47, Matthias St. Pierre wrote: >> >> >> On 30.07.19 12:43, Kurt Roeckx wrote: >>> >>> I currently fail to see how that's a problem, unless that >>> EVP_CIPHER_CTX tries to use a DRBG. >> >> This is what I mean when I say that things have gotten more complicated >> under the hood >> due to the replumbing. To understand the problem, please take at a look of >> the sanitizer >> callstack in >> >> https://github.com/openssl/openssl/issues/9455 >> >> >> Matthias >> >> >> >
Thread sanitiser problems
Bringing the discussions over to the project list. The problem is initially mentioned in #9454. Followup issues with infinite recursion and use after free also get mentioned in #9455. These are addressed by #9477 which clears the flush flag before flushing and removes the dependence on the RAND call. Reproduction seems to require gcc-8, gcc-7 doesn’t have the appropriate thread sanitisation. Overly simplified, the problem boils down to the CTR DRBG needing an AES CTR cipher context to work. When creating the former, a recursive call is made to get the latter. A deadlock results due to a cycle in the locking. The RAND/DRBG code will not be the only place where this occurs. KDF, MAC and some public key operations will suffer a similar issue. This is almost certainly going to hurt moving forward. On to possible mitigations: 1. Make our locks recursive. Add a reference counter and a thread ID to the lock structure. There would be no need to make either use atomic operations. Getting a read lock after a write lock would be considered an extra write lock, handing a write lock while holding a read lock could be problematic. This seems very messy. 2. Return dependent algorithms as part of the registration process. The particular algorithm could be preloaded somehow. I’m not sure how ugly this will become but it will need names (nids) for each possible DRBG type. Thoughts anyone? Any better solutions? Any other solutions? Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Vote on PR
The following vote passed 6 to 0. topic: Accept the changes to the OpenSSL policies as per PR#133 (openssl/web). comment: The definition of trivial being clarified and moved to the web page that the missing CLA note references. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
OSSL_PARAM thought
We’re only starting out, so there isn’t any issue yet. I am wondering if instead of terminating out OSSL_PARAM arrays with an empty element, would it make sense to pass a size and the array? I.e. changing this code sequence (from crypto/evp/digest.c): params[i++] = OSSL_PARAM_construct_size_t(OSSL_DIGEST_PARAM_XOFLEN, &size, NULL); params[i++] = OSSL_PARAM_construct_end(); EVP_MD_CTX_set_params(ctx, params); into: params[i++] = OSSL_PARAM_construct_size_t(OSSL_DIGEST_PARAM_XOFLEN, &size, NULL); EVP_MD_CTX_set_params(ctx, i, params); For fixed arrays OSSL_NELEM would be used instead of the counter variable. There are downsides with both approaches of course and neither jumps out as being obviously superior. To me, at least, it looks like we’re going to have a lot of END’s throughout the codebase. Saving one line many times seems like a win. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: punycode licensing
It seems okay from here too. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 21 Jun 2019, at 11:59 am, Benjamin Kaduk wrote: > > On Thu, Jun 20, 2019 at 12:27:38PM -0400, Viktor Dukhovni wrote: >> On Thu, Jun 20, 2019 at 03:39:10PM +0100, Matt Caswell wrote: >> >>> PR 9199 incorporates the C punycode implementation from RFC3492: >>> >>> https://github.com/openssl/openssl/pull/9199 >>> >> >> I'd be comfortable with relicensing under Apache, while clearly >> indicating the provenance of the code, and indicating that the >> file is also available under the original terms. > > Me, too. > > -Ben
Re: Removing function names from errors (PR 9058)
I’m behind ditching the function identifier #defines but not their text names.
The varargs based error function seems like a good idea too. It would be nicer
if the ERR_raise_error function included the file and line information without
them needing to be specified. As an exercise for the reader, I’ll skip any
implementation details but will note that it is possible without using variadic
macros.
Pauli
Hint:
#define ERR_raise_error ERR_raise_error_internal(__FILE__, __LINE__, __FUNC__)
int (*)(int, const char *, ...) ERR_raise_error_internal(const char *, int,
const char *);
--
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
> On 14 Jun 2019, at 12:04 pm, Viktor Dukhovni
> wrote:
>
> On Wed, Jun 12, 2019 at 05:51:44AM +0200, Richard Levitte wrote:
>
>> A discussion point in that PR is whether it's still interesting to
>> keep information on the system function/callback that was called when
>> we're reporting a system error, i.e. this type of error report:
>>
>>SYSerr(SYS_F_FSTAT, errno);
>>
>> (incidently, there's another PR, 9072, which changes those to
>> 'SYSerr("fstat", errno)')
>>
>> So, the main points of discussion seem to be:
>>
>> - should we remove function indicators in our error reporting?
>> - should we make an exception for system errors (SYSerr())?
>
> Bottom line, my take is that function indicators should stay, but
> string names would be more convenient all around than numeric codes.
>
> --
> Viktor.
Start up entropy gathering
Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on boot and first boot problems. Comment: PR#9084 removed such mitigation due to the negative side effects. This vote has passed (two for, four abstain, none against): The discussion now needs to turn to how this can be done. The mitigation is for older Linux kernels only (and perhaps old Unix machines of other flavours). So no Windows, getentropy(3), …. I have a suggestion. Two kind of. Use a volatile mechanism for communicating between processes — a file in /tmp, a block of shared memory or whatever. On system boot, the first instance of libcrypto that attempts to read /dev/urandom, first locks the shared resource and reads one byte from /dev/random. It uses the shared resource to indicate that the read was successful and continues seeding from /dev/urandom. Other instances of libcrypto start up, see that the shared resource is properly initialised and go straight to using /dev/urandom. On system shutdown/reboot, the communication mechanism disappears and the entire process will repeat on the next boot. There are some fine details but this is the gist of it. This is more difficult to describe than code: see #9152 <https://github.com/openssl/openssl/pull/9152>. Notes: 1. Reading the byte from /dev/random avoid problems on systems that cannot select(2) on /dev/random. 2. Early start up of /libcrypto will block until entropy is available (and consume eight bits of same). Later starts have the overhead of the locking and resource access. Importantly, no entropy will be drained from the system pool after start up. The second suggestion is broadly similar but requires a file containing entropy that persists across reboots. This alternative requires a more management: the entropy file once read needs to be rewritten immediately (and ideally on shutdown as well). It also introduces a new attack vector against the entropy storage. It also isn’t possible to skip the entropy file read/rewrite sequence because it is impossible to determine if /dev/urandom has actually been seeded. I’ve not attempted to code this, persistent files containing seed material potentially introduce other problems. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT
The devices where I’ve experienced long start up delays didn’t all have a HRNG. The rng-tools wouldn’t have achieved anything. ColdFire based routers running a 2.0 Linux kernel were the first time I personally saw this one. uClinux still supported 2.0 kernels last I checked — they are very small and relatively fast. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 8 Jun 2019, at 5:25 am, Kurt Roeckx wrote: > > On Fri, Jun 07, 2019 at 03:08:24PM -0400, Viktor Dukhovni wrote: >>> On Jun 7, 2019, at 2:41 PM, Kurt Roeckx wrote: >>> >>>> This is not the sort of thing to bolt into the kernel, but is not >>>> unreasonable for systemd and the like. >>> >>> The kernel actually already does this in recent versions, if >>> configured to do it. >> >> We're talking about what to do with for older kernels. > > For older kernels you install rng-tools that feeds the hwrng in > the kernel. > > > Kurt >
VOTE Apply PR#9084 reverting DEVRANDOM_WAIT
This vote has been closed, it passed 5 votes to 2 with no abstentions. Up for discussion is the text of the next vote. I’m proposing this: Topic: The OpenSSL 3.0.0 release will include mitigation for the low entropy on boot and first boot problems. Comment: PR#9084 removed such mitigation due to the negative side effects. I’ll make this formal in a day or so, so if anyone wants to suggest alternative wording, that’s the time line. The vote text is the “topic” line, the comment is explanatory only. Note: I’m not mentioning the mechanism used, that still needs to be decided on. This is just saying that 3.0.0 *will* have some mechanism. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT
Storing entropy has risks — cloning attacks, reading the file attacks, etc. It is better trying than doing nothing. Counting such stored entropy as additional data is good but many systems have no alternative entropy available. I have had to introduce such storing schemes (with generation during fulfilment) when devices would take well over ten minutes to find enough entropy to generate their initial (long term) key material. There are ways to mitigate the reuse risk — lock the entropy file, read it, add the contents to the kernel, regenerate the file and finally unlock it. Plus regenerating the contents and storing them on shutdown (if not periodically). A cloned VM would still be vulnerable (at least for a while). Also agreed for an option to disable this. I’d prefer that it is on by default (i.e. secure by default and requiring a decision to not be secure). I’m expecting a somewhat lively discussion about a sensitive topic :) Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 7 Jun 2019, at 6:18 pm, Tomas Mraz wrote: > > On Fri, 2019-06-07 at 18:03 +1000, Dr Paul Dale wrote: >> Mark, I agree that the project area would be better for this >> discussion. >> >> For those not involved, this revolves around the vote to merge >> PR#9084 and possible alternate mitigations for the underlying >> problem. >> >> >> Tim’s explanatory message is: >> >>> For the record, I think we need to do a better job of tackling the >>> underlying issue. >>> Dealing with systems on first boot with no entropy is an issue for >>> a class of systems. >>> However we should be distinguishing between contexts where we have >>> decent seeding material (from having had reasonable entropy) from >>> those where we haven't. >>> We also should be able to persist seeding across boots and have >>> recommendations for how vendors should do that. >>> We should be able to fall back to handling it at the user level >>> (i.e. non-root user, persisted in the file system). >>> We need knowledge of install state (for FIPS) and that can also act >>> as a first-boot state effectively. >>> >>> And this applies equally to getentropy usage - it is only a problem >>> in very limited contexts where blocking actually makes sense to >>> perform. >>> That the kernel has no additional entropy available is not >>> necessarily an issue - except in a very limited context (first time >>> boot with no other entropy available from previous contexts). > > The previous context can easily provide false entropy - for example due > to the booting system being a cloned virtual machine. Or boot image for > a small router device which contains such saved entropy which is > distributed to millions of devices worldwide. > > This was the reason why systemd guys rejected the idea to actually seed > the kernel rng with such entropy with the call that counts this seed > entropy. They just use it as additional seed with no accounting for the > entropy it would provide if it was not reused. > >> >> Viktor replied: >> >>> I just want to make sure we don't lock ourselves in to a single >>> potentially clumsy solution in the library. Various strategies >>> may be appropriate depending on the platform, and ultimately the >>> cooperation of the system administrator to enable the required >>> mechanisms. >>> >>> Potential additional sources for initial entropy on systems >>> without getrandom(2) might include: >>> >>> RDSEED/RDRAND >>> TPM (or Virtualized TPM which is sometimes better!) >>> RNG state saved across reboots. >>> ... >>> >>> This requires knowledge of various platforms, and potential >>> help from the platform vendors. It might, for example, make >>> sense to delegate initial seeding to systemd on platforms >>> that use systemd, and work with the systemd maintainers to >>> provide a set of sensible entropy sources for initial boot. >>> >>> Exposing a decent RNG to virtual guests and containers is >>> would be another area of focus. > > From the point of view of distribution maintainer of OpenSSL I would > say what we had in 1.1.1 before the introduction of DEVRANDOM_WAIT had > no real problems for us. > > However I can understand that when OpenSSL is built and installed > separately by application vendors on some legacy system that they do > not have control of they might appreciate that the library itself tries > to
Re: VOTE Apply PR#9084 reverting DEVRANDOM_WAIT
Mark, I agree that the project area would be better for this discussion. For those not involved, this revolves around the vote to merge PR#9084 and possible alternate mitigations for the underlying problem. Tim’s explanatory message is: > For the record, I think we need to do a better job of tackling the underlying > issue. > Dealing with systems on first boot with no entropy is an issue for a class of > systems. > However we should be distinguishing between contexts where we have decent > seeding material (from having had reasonable entropy) from those where we > haven't. > We also should be able to persist seeding across boots and have > recommendations for how vendors should do that. > We should be able to fall back to handling it at the user level (i.e. > non-root user, persisted in the file system). > We need knowledge of install state (for FIPS) and that can also act as a > first-boot state effectively. > > And this applies equally to getentropy usage - it is only a problem in very > limited contexts where blocking actually makes sense to perform. > That the kernel has no additional entropy available is not necessarily an > issue - except in a very limited context (first time boot with no other > entropy available from previous contexts). Viktor replied: > I just want to make sure we don't lock ourselves in to a single > potentially clumsy solution in the library. Various strategies > may be appropriate depending on the platform, and ultimately the > cooperation of the system administrator to enable the required > mechanisms. > > Potential additional sources for initial entropy on systems > without getrandom(2) might include: > > RDSEED/RDRAND > TPM (or Virtualized TPM which is sometimes better!) > RNG state saved across reboots. > ... > > This requires knowledge of various platforms, and potential > help from the platform vendors. It might, for example, make > sense to delegate initial seeding to systemd on platforms > that use systemd, and work with the systemd maintainers to > provide a set of sensible entropy sources for initial boot. > > Exposing a decent RNG to virtual guests and containers is > would be another area of focus. My suggestion as a fallback would be Stephan Müller’s CPU Jitter <http://chronox.de/jent/doc/CPU-Jitter-NPTRNG.html>. He’s collected a large corpus of data from many processors and the scheme works relatively quickly. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 7 Jun 2019, at 5:19 pm, Mark J Cox wrote: > > Could we have this more detailed discussion on -project? > > Mark
Re: OSSL_PARAMs
The OSSL_PARAM structure needs to be visible and not subject to change.
Providers shouldn’t necessarily have a dependency on functions from libcrypto.
Pauli
--
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
> On 5 Jun 2019, at 12:47 pm, SHANE LONTIS wrote:
>
>
>
>> On 5 Jun 2019, at 12:34 pm, Richard Levitte > <mailto:levi...@openssl.org>> wrote:
>>
>> Aside from the discussion below, if there's one thing I would like to
>> change, it the double indirection for the _PTR data types. The data
>> types could still be used to indicate that the value isn't short
>> lived, but could possibly change names to something like
>> OSSL_PARAM_UTF8_CSTRING and OSSL_PARAM_OCTET_CSTRING (C for Constant).
>>
>> On Wed, 05 Jun 2019 01:18:56 +0200,
>> Dr Paul Dale wrote:
>>> Shane’s major complaints are about the indirection the OSSL_PARAM structure
>>> forces — for integers
>>> and return lengths and the necessity of allocating additional memory in
>>> parallel with the
>>> OSSL_PARAM.
>>>
>>> The extra indirection was intended to support const arrays of OSSL_PARAM,
>>> which turn out to be a
>>> rarity because they aren’t thread safe.
>>
>> The reason why we have this issue is our base C language version
>> choice. C90 doesn't allow this construct:
>>
>>int foo(whatever)
>>{
>>int haha = 0;
>>const OSSL_PARAM params[] = {
>>{ 'foo', OSSL_PARAM_INTEGER, &haha, sizeof(haha), NULL },
>>{ NULL, 0, NULL, 0, NULL }
>>};
>>
>>...
>>}
>>
> The above code is great in theory, but it looks like in practice we end up
> dynamically allocating in most cases anyway (via the construct_ methods).
> And if this is the normal use case then OSSL_PARAMS could be made opaque and
> only accessed by API’s, then the argument about adding
> extra types later on should also disappear?
>
>
>> Because the compiler for that language version isn't allowed to emit
>> code to use '&haha' in an inititializer. Newer C language versions
>> allow this.
>>
>> So while this is an issue for *us*, it isn't necessarily an issue for
>> our users, all depending on what C language version they use.
>>
>>> With most OSSL_PARAM structure being dynamically created,
>>> the need for the indirection seems redundant. E.g. could the return length
>>> be moved into
>>> OSSL_PARAM? I think so.
>>
>> The design was not only to be able to have nice compile time
>> initialization, but also to be able to pass the array as 'const
>> OSSL_PARAM *', i.e. an indication to the recipient that the array
>> itself should never be modified (less chance of compromise). Maybe
>> that's overly paranoid, but that was a line of thinking.
>>
>>> Moving integral values into the structure is more difficult because BIGNUMs
>>> will always need to be
>>> references. Allocating additional memory will still be required. I’ve got
>>> three obvious
>>> solutions:
>>>
>>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when
>>> the structure is
>>> destroyed or
>>
>> It's actually perfectly possible to do this today. We already have
>> this pointer, it's called 'data'.
>>
>>> 2. have a block of data in the OSSL_PARAM structure that can be used for
>>> native types
>>> (OSSL_UNION_ALIGN works perfectly for this) or
>>
>> My major concern with that, apart from having to modify the OSSL_PARAM
>> items themselves¸ is that some time in the future, we will want to add
>> another native type that's larger, which means we modify the size of a
>> OSSL_PARAM. It's a public structure, so that can't be treated
>> lightly.
>>
>> Also, with a union of native types, we're losing uniformity on MSB
>> first platforms. Having an exact 1:1 integer size match will be
>> crucial, and that complicates the code quite a bit... not to mention
>> that we have a compatibility problem as soon as one end has a new
>> native type in the union and the other doesn't.
>> (one would imagine that simply using uintmax_t would cover all integer
>> sizes apart from BIGNUM, but the potential size change of that type
>> with newer compilers make such a choice precarious)
>>
>>> 3. add a flag field to the OSSL_
Re: OSSL_PARAMs
-- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 12:47 pm, Richard Levitte wrote: > But you're talking about allocating the whole OSSL_PARAM array on the > heap, aren't you? While not structly opposed (even though I think > that's wasteful), I think this should be done with a separate set of > functions (intuitively, having names with '_alloc_' rather than > '_construct_' could be an idea). Not the whole OSSL_PARAM array, just the data pointed to by (some) of the elements of one. _alloc_ instead of _construct_ makes sense and is better than NULL. I’m fine with `OSSL_PARAM params[20]` in code and filling as many slots as desired. To do entirely heap allocated arrays, we’d want to go via a stack (which is something we shouldn’t require providers to depend on) or use a linked list of some kind. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: OSSL_PARAMs
Richard wrote: -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > So while this is an issue for *us*, it isn't necessarily an issue for > our users, all depending on what C language version they use. Supporting things *we* can’t use seems a little odd. The alternative construction is about as onerous (the construct calls). On stack arrays won’t provide much performance benefit. Static arrays would but without thread safety. >> With most OSSL_PARAM structure being dynamically created, >> the need for the indirection seems redundant. E.g. could the return length >> be moved into >> OSSL_PARAM? I think so. > > The design was not only to be able to have nice compile time > initialization, but also to be able to pass the array as 'const > OSSL_PARAM *', i.e. an indication to the recipient that the array > itself should never be modified (less chance of compromise). Maybe > that's overly paranoid, but that was a line of thinking. This is a better reason, not that I think “const” is all that useful here. An aside: having a const struct that has a pointer to non-const memory isn’t entirely obvious to many. This is a public API, make it as simple as necessary. >> Moving integral values into the structure is more difficult because BIGNUMs >> will always need to be >> references. Allocating additional memory will still be required. I’ve got >> three obvious >> solutions: >> >> 1. include a void * in the OSSL_PARAM structure that needs to be freed when >> the structure is >> destroyed or > > It's actually perfectly possible to do this today. We already have > this pointer, it's called 'data’. And how is a pointer known to be malloced or not? I’m trying to make this easy for users without losing the efficiency that is possible if it is required somewhere. I’m looking at making this kind of code work: OSSL_PARAMS params[10]; int n = 0, max_len; char my_size[20]; scanf(“%d”, &max_len); params[n++] = OSSL_PARAM_construct_utf8_string(“size”, &my_size, sizeof(my_size), NULL); params[n++] = OSSL_PARAM_construct_utf8_string(“name”, NULL, max_len, NULL); params[n++] = OSSL_PARAM_construct_end(); … OSSL_PARAM_deconstruct_all(params); It is a contrived case but I think it would make using the params easier. >> 2. have a block of data in the OSSL_PARAM structure that can be used for >> native types >> (OSSL_UNION_ALIGN works perfectly for this) or > > My major concern with that, apart from having to modify the OSSL_PARAM > items themselves¸ is that some time in the future, we will want to add > another native type that's larger, which means we modify the size of a > OSSL_PARAM. It's a public structure, so that can't be treated > lightly. This is a valid concern. OSSL_UNION_ALIGN isn’t appropriate. Likewise, uintmax_t isn’t binary compatible. > If you're thinking that the receiving side should free certain values, > then you need to pass a pointer to the routine to be used to free the > value rather than just a flag. I agree, this is a bad idea. The API isn’t easy to use at the moment. It is also error prone as soon as something complex is encountered (which we haven’t yet). Shane struggled to figure this out and get it working when trying the KDFs and he is far more experienced than most. I’ll see if I can get some kind of diff or PR together. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: OSSL_PARAMs
For a minimally invasive change, the various OSSL_PARAM_construct_ calls could be modified to allocate space for the parameter and its return size if NULL is passed for either. There would need to be some indication of this and a cleanup array call. Done properly, this would permit the same efficiency on the owner’s side but would also simplify OSSL_PARAM array creation for those who aren’t as concerned. The theoretical const array of OSSL_PARAM would still be possible. Is this workable or should something more significantly different be used before things freeze with the 3.0 release? Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 10:50 am, Dr Paul Dale wrote: > > I thought the references were to allow const arrays of OSSL_PARAM to be > viable. > > A quick check through the code reveals these in test and doc only. There are > two instances of OSSL_PARAM arrays being declared in evp, both add the > pointed to variable after creation, both only have two elements (the integer > and the terminator) and both are stack allocated. I.e. there is currently is > no example of the use case for which the indirection is present :( > > > Pauli > -- > Dr Paul Dale | Cryptographer | Network Security & Encryption > Phone +61 7 3031 7217 > Oracle Australia > > > >> On 5 Jun 2019, at 10:31 am, SHANE LONTIS > <mailto:shane.lon...@oracle.com>> wrote: >> >> I presume the reference approach was used to solve the issue of who actually >> owns/free's the data. >> >> >>> On 5 Jun 2019, at 9:18 am, Dr Paul Dale >> <mailto:paul.d...@oracle.com>> wrote: >>> >>> Shane’s major complaints are about the indirection the OSSL_PARAM structure >>> forces — for integers and return lengths and the necessity of allocating >>> additional memory in parallel with the OSSL_PARAM. >>> >>> The extra indirection was intended to support const arrays of OSSL_PARAM, >>> which turn out to be a rarity because they aren’t thread safe. With most >>> OSSL_PARAM structure being dynamically created, the need for the >>> indirection seems redundant. E.g. could the return length be moved into >>> OSSL_PARAM? I think so. >>> >>> Moving integral values into the structure is more difficult because BIGNUMs >>> will always need to be references. Allocating additional memory will still >>> be required. I’ve got three obvious solutions: >>> >>> 1. include a void * in the OSSL_PARAM structure that needs to be freed when >>> the structure is destroyed or >>> 2. have a block of data in the OSSL_PARAM structure that can be used for >>> native types (OSSL_UNION_ALIGN works perfectly for this) or >>> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value >>> needs to be freed. >>> >>> The memory allocation comes to the for when reading e.g. a file and >>> extracting data — either the reader needs a lot of local variables to hold >>> everything or it has to allocated for each. The file’s data is transient >>> in memory. >>> >>> >>> For the most part, the receiver side APIs seem reasonable. It is the >>> owning side that has the complications. >>> >>> I think I might be able come up with some owner side routines that assist >>> here but allowing changes to the params structure would be far easier. >>> >>> >>> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl >>> functions if not ctrl as well (subject to backward compatibility concerns). >>> >>> >>> Pauli >>> -- >>> Dr Paul Dale | Cryptographer | Network Security & Encryption >>> Phone +61 7 3031 7217 >>> Oracle Australia >>> >>> >>> >>>> On 4 Jun 2019, at 11:26 pm, Richard Levitte >>> <mailto:levi...@openssl.org>> wrote: >>>> >>>> On Tue, 04 Jun 2019 14:57:00 +0200, >>>> Salz, Rich wrote: >>>>> >>>>> >>>>>> Part of the idea was that this would be a means of communication >>>>>between application and provider, just like controls are with >>>>>libcrypto sub-systems. >>>>> >>>>> >>>>> I can probably find the email thread (or maybe it was a GitHub >>>>> comment on my proposal for params), where you said, quite >>>>> definitively, that
Re: OSSL_PARAMs
I thought the references were to allow const arrays of OSSL_PARAM to be viable. A quick check through the code reveals these in test and doc only. There are two instances of OSSL_PARAM arrays being declared in evp, both add the pointed to variable after creation, both only have two elements (the integer and the terminator) and both are stack allocated. I.e. there is currently is no example of the use case for which the indirection is present :( Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 5 Jun 2019, at 10:31 am, SHANE LONTIS wrote: > > I presume the reference approach was used to solve the issue of who actually > owns/free's the data. > > >> On 5 Jun 2019, at 9:18 am, Dr Paul Dale > <mailto:paul.d...@oracle.com>> wrote: >> >> Shane’s major complaints are about the indirection the OSSL_PARAM structure >> forces — for integers and return lengths and the necessity of allocating >> additional memory in parallel with the OSSL_PARAM. >> >> The extra indirection was intended to support const arrays of OSSL_PARAM, >> which turn out to be a rarity because they aren’t thread safe. With most >> OSSL_PARAM structure being dynamically created, the need for the indirection >> seems redundant. E.g. could the return length be moved into OSSL_PARAM? I >> think so. >> >> Moving integral values into the structure is more difficult because BIGNUMs >> will always need to be references. Allocating additional memory will still >> be required. I’ve got three obvious solutions: >> >> 1. include a void * in the OSSL_PARAM structure that needs to be freed when >> the structure is destroyed or >> 2. have a block of data in the OSSL_PARAM structure that can be used for >> native types (OSSL_UNION_ALIGN works perfectly for this) or >> 3. add a flag field to the OSSL_PARAM to indicate that the referenced value >> needs to be freed. >> >> The memory allocation comes to the for when reading e.g. a file and >> extracting data — either the reader needs a lot of local variables to hold >> everything or it has to allocated for each. The file’s data is transient in >> memory. >> >> >> For the most part, the receiver side APIs seem reasonable. It is the owning >> side that has the complications. >> >> I think I might be able come up with some owner side routines that assist >> here but allowing changes to the params structure would be far easier. >> >> >> I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl >> functions if not ctrl as well (subject to backward compatibility concerns). >> >> >> Pauli >> -- >> Dr Paul Dale | Cryptographer | Network Security & Encryption >> Phone +61 7 3031 7217 >> Oracle Australia >> >> >> >>> On 4 Jun 2019, at 11:26 pm, Richard Levitte >> <mailto:levi...@openssl.org>> wrote: >>> >>> On Tue, 04 Jun 2019 14:57:00 +0200, >>> Salz, Rich wrote: >>>> >>>> >>>>> Part of the idea was that this would be a means of communication >>>>between application and provider, just like controls are with >>>>libcrypto sub-systems. >>>> >>>> >>>> I can probably find the email thread (or maybe it was a GitHub >>>> comment on my proposal for params), where you said, quite >>>> definitively, that this was *not* a general-purpose mechanism but >>>> rather a way to expose the necessary internals for opaque objects >>>> like RSA keys. >>> >>> Either I misunderstood what you said at the time, or you misunderstood >>> what I said... there's definitely a disconnect here somewhere. >>> >>> What I wonder is why it should be exclusively only one of those >>> options? >>> >>> Either way, the OSSL_PARAM is defined publically and openly (i.e. >>> non-opaque), and we currently have the following functions in the >>> public API: >>> >>>EVP_MD_CTX_set_params >>>EVP_MD_CTX_get_params >>>OSSL_PROVIDER_get_params >>> >>> I fully expect that more will come. I have a branch where I've >>> EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if >>> EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before >>> long (I'm actually rather surprised they haven't already), and I'm >>> absolutely sure we will se
Re: OSSL_PARAMs
Shane’s major complaints are about the indirection the OSSL_PARAM structure forces — for integers and return lengths and the necessity of allocating additional memory in parallel with the OSSL_PARAM. The extra indirection was intended to support const arrays of OSSL_PARAM, which turn out to be a rarity because they aren’t thread safe. With most OSSL_PARAM structure being dynamically created, the need for the indirection seems redundant. E.g. could the return length be moved into OSSL_PARAM? I think so. Moving integral values into the structure is more difficult because BIGNUMs will always need to be references. Allocating additional memory will still be required. I’ve got three obvious solutions: 1. include a void * in the OSSL_PARAM structure that needs to be freed when the structure is destroyed or 2. have a block of data in the OSSL_PARAM structure that can be used for native types (OSSL_UNION_ALIGN works perfectly for this) or 3. add a flag field to the OSSL_PARAM to indicate that the referenced value needs to be freed. The memory allocation comes to the for when reading e.g. a file and extracting data — either the reader needs a lot of local variables to hold everything or it has to allocated for each. The file’s data is transient in memory. For the most part, the receiver side APIs seem reasonable. It is the owning side that has the complications. I think I might be able come up with some owner side routines that assist here but allowing changes to the params structure would be far easier. I kind of like using the OSSL_PARAM arrays as a replacement for string ctrl functions if not ctrl as well (subject to backward compatibility concerns). Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 4 Jun 2019, at 11:26 pm, Richard Levitte wrote: > > On Tue, 04 Jun 2019 14:57:00 +0200, > Salz, Rich wrote: >> >> >>> Part of the idea was that this would be a means of communication >>between application and provider, just like controls are with >>libcrypto sub-systems. >> >> >> I can probably find the email thread (or maybe it was a GitHub >> comment on my proposal for params), where you said, quite >> definitively, that this was *not* a general-purpose mechanism but >> rather a way to expose the necessary internals for opaque objects >> like RSA keys. > > Either I misunderstood what you said at the time, or you misunderstood > what I said... there's definitely a disconnect here somewhere. > > What I wonder is why it should be exclusively only one of those > options? > > Either way, the OSSL_PARAM is defined publically and openly (i.e. > non-opaque), and we currently have the following functions in the > public API: > >EVP_MD_CTX_set_params >EVP_MD_CTX_get_params >OSSL_PROVIDER_get_params > > I fully expect that more will come. I have a branch where I've > EVP_MAC_CTX_set_params, for example, and I wouldn't be surprised if > EVP_CIPHER_CTX_set_params and EVP_CIPHER_CTX_get_params appear before > long (I'm actually rather surprised they haven't already), and I'm > absolutely sure we will see similar functions for asymmetric > algorithms. > >> What changed your mind? >> >> Perhaps not surprisingly, I agree with Shane's assessment and am >> strongly opposed to the project foisting this on everyone at this >> time. @DavidBen, your thoughts? > > Maybe we're reading differently, I didn't see Shane being opposed to > parameter passing in this way per se, just the exact form of the > OSSL_PARAM structure, which is different. > > Cheers, > Richard > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/
OSSL_PARAMs
A question for the committers and beyond. Should the OSSL_PARAM concept be used for more than just the libcrypto ⇿ provider interface? The one that springs to mind is a more general form of the various ctrl calls, there are some string ctrl calls that attempt to do the same thing in a less structured manner. Thoughts? Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia
Re: Any timeframe for the 1.1.1c release?
This seems reasonable to me. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 6 May 2019, at 5:40 pm, Richard Levitte wrote: > > Our last update release was by the end of February. With our usual > 3-ish month interval, end of May would be quite suitable, methinks. > > I propose Tuesday May 28th as a release date for 1.1.1c. > > Cheers, > Richard > > > > There should perhaps be a 1.1.1c soonish... There are indeed many useful > improvements committed, but not yet released. > > -- > Viktor. > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > >
Re: Issues and pull requests are largely getting ignored
I agree with Matt. He and Richard are doing a great job but cannot quite keep up. They are doing a fantastic job given the time available to them. I’ve been a bit slack the last two weeks and will be so for the next four or so but I’ll attempt to catch up. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 26 Mar 2019, at 7:53 pm, Matt Caswell wrote: > > > > On 25/03/2019 20:10, Matthew Lindner wrote: >> Hello OpenSSL Team, >> >> The issues and pull requests on github are largely getting ignored, I >> know the team is busy on the new release but please spend some time on >> these as well. > > I don't think this is a fair characterisation. I see all posts to issues and > PRs > in github for openssl and I see *lots* of activity. > > If I use the github "insights" page I see that over the last month 67 new > issues > were raised and 46 were closed. However those numbers are skewed by issues in > very recent days (its shortly after a weekend and neither Richard or myself > were > around yesterday). Over the last week 23 new issues were raised and 8 were > closed. Subtracting one from the other I see that for the period 1 week ago > to 1 > month ago 44 new issues were raised and 38 were closed. > > So the real problem there is a mismatch between the opening rate and the > closing > rate, i.e. it is NOT that we are ignoring these issues. I see it more as a > resource issue - we are seeing issues raised at a greater rate than we have > resources to handle. The best answer to that IMO is to somehow encourage a > greater participation from the community in helping us to keep on top of these > things. Not sure how best to achieve that though. > > I can't do the same analysis for PRs because github doesn't detect properly > when > we merge PRs (because we don't use the github facility for doing that). I > would > expect to see somewhat similar numbers though. > > > Matt
Re: Thoughts on OSSL_ALGORITHM
I’ve no issue having a provider data field there. It will be useful for more
than just this (S390 AES e.g. holds data differently to other implementations).
I also don’t think forcing separate functions is a big problem — most providers
will only implement one or two algorithm families which will help control the
redundancy.
I don’t think we should be doing a string lookup every time one of these is
called.
Of the three, the provider data feels clean and unique functions fast.
I’d like to avoid mandating another level of indirection (it’s slow), which is
a risk with provider data.
My thought: add the provider data field. Use that when it can be done
directly, use unique functions otherwise.
The example with key and iv lengths would be a direct use. Code that dives
through a function pointer or a switch statement would be an example of not.
Pauli
--
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
> On 23 Mar 2019, at 1:45 am, Matt Caswell wrote:
>
> Currently we have the OSSL_ALGORITHM type defined as follows:
>
> struct ossl_algorithm_st {
>const char *algorithm_name; /* key */
>const char *property_definition; /* key */
>const OSSL_DISPATCH *implementation;
> };
>
> I'm wondering whether we should add an additional member to this structure: a
> provider specific handle. i.e.
>
> struct ossl_algorithm_st {
>const char *algorithm_name; /* key */
>const char *property_definition; /* key */
>const OSSL_DISPATCH *implementation;
>void *handle;
> };
>
> The reason to do this is because providers are likely to want to share the
> same
> implementation across multiple algorithms, e.g. the init/update/final
> functions
> for aes-128-cbc are likely to look identical to aes-256-cbc with the only
> difference being the key length. A provider could use the handle to point to
> some provider side structure which describes the details of the algorithm (key
> length, IV size etc). For example in the default provider we might have:
>
> typedef struct default_alg_handle_st {
>int nid;
>size_t keylen;
>size_t ivlen;
> } DEFAULT_ALG_HANDLE;
>
> DEFAULT_ALG_HANDLE aes256cbchandle = { NID_aes_256_cbc, 32, 16 };
> DEFAULT_ALG_HANDLE aes128cbchandle = { NID_aes_128_cbc, 16, 16 };
>
> static const OSSL_ALGORITHM deflt_ciphers[] = {
>{ "AES-256-CBC", "default=yes", aes_cbc_functions, &aes256cbchandle },
>{ "AES-128-CBC", "default=yes", aes_cbc_functions, &aes128cbchandle },
>{ NULL, NULL, NULL }
> };
>
> Then when the "init" function is called (or possibly at newctx), the core
> passes
> as an argument the provider specific handle associated with that algorithm.
>
> An alternative is for the provider to pass the algorithm name instead, but
> this
> potentially requires lots of strcmps to identify which algorithm we're dealing
> with which doesn't sound particularly attractive.
>
> A second alternative is for each algorithm to always have unique functions
> (which perhaps call some common functions underneath). This sounds like lots
> of
> unnecessary redundancy.
>
> Thoughts?
>
> Matt
Re: [openssl-project] inline functions
Richard wrote: > Not really, since they are static inline. This is by design, that for any > file you want to use a safestack in, you just start with a DEFINE_ line. The > mistake we did was to leave a few common ones in the safestack header file. > (same thing for lhash) Which means we’ve a compatibility issue. The functions are in a public header, they can be used by any application. We need to continue supporting such use. Asking a user to add a DEFINE_ line is API breaking. I would be pro making such a change but we’d need to accept the consequences. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] inline functions
Isn’t this going to run into the same issues?
The DEFINE_ expansions need to be in their own .c file (one file with all or
one file per, I’m not fussed much but the latter seems nicer).
The DEFINE_ macros need to stay in the header for compatibility. To handle the
external linkage, a new macro that declares the function prototypes should be
introduced. I.e.:
# define DECLARE_LHASH_OF(type) \
LHASH_OF(type) * ln_##type##_new(unsigned long (*)(const type *), int
(*)(const type *, const type *)); \
void lh_##type##_free(LHASH_OF(type) *); \
…
# define DEFINE_LHASH_OF(type) \
DECLARE LHASH_OF(type); \
LHASH_OF(type) { union lh_##type##_dummy { void* d1; unsigned long d2; int
d3; } dummy; }; \
static ossl_inline LHASH_OF(type) * \
lh_##type##_new(unsigned long (*hfn)(const type *), \
int (*cfn)(const type *, const type *)) \
{ \
return (LHASH_OF(type) *) \
OPENSSL_LH_new((OPENSSL_LH_HASHFUNC)hfn, (OPENSSL_LH_COMPFUNC)cfn);
\
} \
static ossl_inline void lh_##type##_free(LHASH_OF(type) *lh) \
{ \
OPENSSL_LH_free((OPENSSL_LHASH *)lh); \
} \
…
The headers can then use the DECLARE_LHASH_OF macro to prototype the functions.
The .c file uses the DEFINE_LHASH_OF macro to create them.
I chose lhash here because it is the simpler of the two, safestack has more
options and is a bit more convoluted. I’m willing to make a stab at a PR for
this.
Pauli
--
Dr Paul Dale | Cryptographer | Network Security & Encryption
Phone +61 7 3031 7217
Oracle Australia
> On 28 Jan 2019, at 4:56 am, Richard Levitte wrote:
>
> On Sun, 27 Jan 2019 18:51:58 +0100,
> Viktor Dukhovni wrote:
>>
>>> On Jan 27, 2019, at 5:33 AM, Tim Hudson wrote:
>>>
>>> Tim - I think inline functions in public header files simply shouldn't be
>>> present.
>>
>> I think they have their place, and we should try to make them more portable
>> to less capable toolchains as needed.
>
> A simple way to handle it is to move the DEFINE_ expansions to another
> header file. I suggest common_stacks.h and common_lhashes.h
>
> At the very least for 3.0.0. For 1.1.1, I have no idea...
>
> Cheers,
> Richard
>
> --
> Richard Levitte levi...@openssl.org
> OpenSSL Project http://www.openssl.org/~levitte/
> ___
> openssl-project mailing list
> openssl-project@openssl.org
> https://mta.openssl.org/mailman/listinfo/openssl-project
___
openssl-project mailing list
openssl-project@openssl.org
https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] inline functions
Yes, those are the problematic cases. I think that making the symbols weak is “good enough” for the moment. Longer term, we could do with a better solution. Moving the implementations into another file is one option. There is another longer term alternative: migrate OpenSSL away from lhash and safestack by introducing new internal functionality that replaces them. We can’t remove either in case users user them but we could stop using them ourselves. Lhash is based on a clever hash table that dynamically resizes (both increasing and decreasing) and amortises the rehashing costs over time. If the OpenSSL source code is looked through, there are relatively few removals from the hash table. I.e. the size decrease isn’t used much, if at all. Likewise, the rehashing checks one bit in the hash for each item and moves it into one of two lists based based on the result. I.e. rehashing should be a fast operation and amortising it doesn’t feel like a win. On the down side, lhash runs with a fairly high level of loading (many entries relative to table size) and its collision resolution is a linked list (i.e. O(n) worst case instead of O(1)). There have also been improvements in hash technology since the lhash algorithm was created — cache coherent algorithms and lock free ones spring to mind. Safestack isn’t a stack anymore. It is used as a vector, an array substitute, a queue and more. I don’t think I’ve seen it used as a stack but it probably is. We should have separate data structures for the different uses, each optimised for its specific usage. This would be a long path (and I’m hijacking this thread a bit), but it is something I’ve been wanting to do for a while now. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 27 Jan 2019, at 10:58 pm, Richard Levitte wrote: > > You're talking about these lines from safestack.h: > >DEFINE_SPECIAL_STACK_OF(OPENSSL_STRING, char) >DEFINE_SPECIAL_STACK_OF_CONST(OPENSSL_CSTRING, char) >DEFINE_SPECIAL_STACK_OF(OPENSSL_BLOCK, void) > > and these from lhash.h: > >DEFINE_LHASH_OF(OPENSSL_STRING); >DEFINE_LHASH_OF(OPENSSL_CSTRING); > > I didn't think of those when looking at the PR, but you're entirely > correct that they are the direct cause of the issue, and should move > to someplace internal. > > Cheers, > Richard > > On Sun, 27 Jan 2019 12:30:07 +0100, > Dr Paul Dale wrote: >> >> >> I’d generally prefer functions over macros — I think that the ctrl calls >> e.g. would be better >> wrapped with function to provide type checking. >> The overhead of a function call is pretty light these days so inline >> functions are difficult to >> justify (as anything except a premature optimisation?). >> >> Both safestack and lhash are problematic cases. The inline functions come >> from macros which I >> view as okay. The problem is that some of these macros are expanded in the >> header for common >> cases (e.g. stack of stings). We could address this by distinguishing >> between the function >> declarations and their instantiation and move the latter into its own C file. >> >> Pauli >> -- >> Dr Paul Dale | Cryptographer | Network Security & Encryption >> Phone +61 7 3031 7217 >> Oracle Australia >> >>On 27 Jan 2019, at 8:33 pm, Tim Hudson wrote: >> >>From https://github.com/openssl/openssl/pull/7721 >> >>Tim - I think inline functions in public header files simply shouldn't be >> present. >>Matt - I agree >>Richard - I'm ambivalent... in the case of stack and lhash, the generated >> functions we made >>static inline expressly to get better C type safety, and to get away from >> the mkstack.pl >> horror. >> >>It would be good to get a sense of the collective thoughts on the topic. >> >>Tim. >> >>___ >>openssl-project mailing list >>openssl-project@openssl.org >>https://mta.openssl.org/mailman/listinfo/openssl-project >> >> >> ___ >> openssl-project mailing list >> openssl-project@openssl.org >> https://mta.openssl.org/mailman/listinfo/openssl-project > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] inline functions
I’d generally prefer functions over macros — I think that the ctrl calls e.g. would be better wrapped with function to provide type checking. The overhead of a function call is pretty light these days so inline functions are difficult to justify (as anything except a premature optimisation?). Both safestack and lhash are problematic cases. The inline functions come from macros which I view as okay. The problem is that some of these macros are expanded in the header for common cases (e.g. stack of stings). We could address this by distinguishing between the function declarations and their instantiation and move the latter into its own C file. Pauli -- Dr Paul Dale | Cryptographer | Network Security & Encryption Phone +61 7 3031 7217 Oracle Australia > On 27 Jan 2019, at 8:33 pm, Tim Hudson wrote: > > From https://github.com/openssl/openssl/pull/7721 > <https://github.com/openssl/openssl/pull/7721> > > Tim - I think inline functions in public header files simply shouldn't be > present. > Matt - I agree > Richard - I'm ambivalent... in the case of stack and lhash, the generated > functions we made static inline expressly to get better C type safety, and to > get away from the mkstack.pl <http://mkstack.pl/> horror. > > It would be good to get a sense of the collective thoughts on the topic. > > Tim. > > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] Review
Thanks, Richard. I’ll merge tomorrow and publish CVE 20181030. Pauli > On 29 Oct 2018, at 8:21 pm, Richard Levitte wrote: > > In message <785270db-e18c-4c5a-a961-765859cd6...@oracle.com> on Mon, 29 Oct > 2018 19:45:36 +1000, Dr Paul Dale said: > >> I’d like a prompt review of #7513 so I can push the second CVE out. >> #7512 is kind of related but not CVE level. > > Done > > -- > Richard Levitte levi...@openssl.org > OpenSSL Project http://www.openssl.org/~levitte/ > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
[openssl-project] Review
I’d like a prompt review of #7513 so I can push the second CVE out. #7512 is kind of related but not CVE level. Pauli ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] Help deciding on PR 6341 (facilitate reading PKCS#12 objects in OSSL_STORE)
I also believe that we shouldn’t be relying on locale, it is a Pandora’s box we don’t want to open. Even claiming that OpenSSL is UTF-8 compliant is probably a stretch (e.g. the isXXX functions aren’t). Saying we accept unsigned eight bit byte inputs and process them unmodified is as far as I’d like to commit to. Pauli > On 2 Jun 2018, at 9:08 am, Viktor Dukhovni wrote: > > > >> On Jun 1, 2018, at 6:47 PM, Richard Levitte wrote: >> >> Ah, forgot one important detail: it is well understood that *all* >> file based objects will get the same requirements, right? That goes >> for anything protected through PKCS#5 as well (good ol' PEM >> encryption, PKCS#8 objects and whatever else I forget...) > > Canonicalize when importing for use with the store API. Not sure > whether wchar_t though, just octet string in UTF-8 seems saner. > That is the password is an opaque byte string, not a character > string in the platform's encoding of i18n strings. > > -- > Viktor. > > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
Re: [openssl-project] Entropy seeding the DRBG
Apologies for the name I’ve been sending under. I don’t represent Oracle of course. A temporary new MUA that isn’t quite doing what I expected. Pauli > On 8 May 2018, at 7:33 pm, Oracle wrote: > > Kurt wrote: > >> The comment about not hashing it is if you want to use the tool to >> do entropy estimation. Hashing it will not increase the entropy, >> but the estimation will be totally wrong. > > >> Passing the hashed data to the drbg as entropy input is fine if >> you already know how much entropy that it contains. > > > This is spot on. Hash the data and it will appear to have eight bits per > byte of entropy regardless of the input. The estimate output from NIST’s > suite will be around 7.8 bits per byte but that’s close enough. The > standards refer to this as “whitening”. It is fine to whiten the entropy > data before passing it to the DRBG but the entropy estimate must be based on > the pre-whitened data. > > > Pauli > > > > > ___ > openssl-project mailing list > openssl-project@openssl.org > https://mta.openssl.org/mailman/listinfo/openssl-project ___ openssl-project mailing list openssl-project@openssl.org https://mta.openssl.org/mailman/listinfo/openssl-project
