RE: BIGNUM library
Is there a specification on the format of a BIGNUM that someone can point me to? Is there a standard encoding/format that everyone adheres to? Or would different libraries have their own encodings? I hope not. Thanks, Ed From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Edward Chan Sent: Monday, April 16, 2007 4:27 PM To: openssl-users@openssl.org Subject: BIGNUM library How easy or difficult would it be to extract just the BIGNUM library from OpenSSL? Are there any documents on how to do this? Thanks, Ed
RE: BIGNUM library
Hello, > Is there a specification on the format of a BIGNUM that someone can > point me to? Probably headers files. > Is there a standard encoding/format that everyone adheres to? Probably not. > Or would different libraries have their own encodings? I hope not. Most bignum libraries use their own formats, in general they are very close (structure with words/limbs, some counters, sign variable ...). Sometimes APIs between libraries are almost the same. > How easy or difficult would it be to extract just the BIGNUM library > from OpenSSL? Are there any documents on how to do this? Very ease, but if you think about bignum library I think GMP is better than extract from OpenSSL. Best regards, -- Marek Marcola <[EMAIL PROTECTED]> __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Thanks for the reply. So I'm a bit confused as to how different crypto packages interoperate. I've been having a problem with MS CryptoAPI doing a DH key exchange with OpenSSL. There are some quirks such as byte order, and a couple other things. But for the most part, I have it working. But occasionally, I get failures which I suspect is due to the BIGNUM format of the pubkey that is exchanged. Has anybody had success getting these 2 libs working together? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Marek Marcola Sent: Tuesday, April 17, 2007 1:55 AM To: openssl-users@openssl.org Subject: RE: BIGNUM library Hello, > Is there a specification on the format of a BIGNUM that someone can > point me to? Probably headers files. > Is there a standard encoding/format that everyone adheres to? Probably not. > Or would different libraries have their own encodings? I hope not. Most bignum libraries use their own formats, in general they are very close (structure with words/limbs, some counters, sign variable ...). Sometimes APIs between libraries are almost the same. > How easy or difficult would it be to extract just the BIGNUM library > from OpenSSL? Are there any documents on how to do this? Very ease, but if you think about bignum library I think GMP is better than extract from OpenSSL. Best regards, -- Marek Marcola <[EMAIL PROTECTED]> __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Tue, Apr 17, 2007 at 08:42:12AM -0700, Edward Chan wrote: > Thanks for the reply. So I'm a bit confused as to how different crypto > packages interoperate. I've been having a problem with MS CryptoAPI > doing a DH key exchange with OpenSSL. There are some quirks such as > byte order, and a couple other things. But for the most part, I have it > working. But occasionally, I get failures which I suspect is due to the > BIGNUM format of the pubkey that is exchanged. Has anybody had success > getting these 2 libs working together? You are confusing BIGNUM which is an internal library structure for performing high precision integer arithmetic, with ASN.1 which is an on the wire format for serialized data. The protocols that exchange public keys, ... are not sending BIGNUM structures around, OpenSSL serializes and de-deserializes BIGNUMs to/from appropriate ASN.1 structures. For example X.509v3 is an ASN.1 data structure that contains some large integers, but they are not in any sense BIGNUMs until OpenSSL parses the ASN.1 and constructs its own (X509 *) structure, in which some of the fields ultimately contain BIGNUM references. The serialized ASN.1 is standard, the in-memory (X509 *) is OpenSSL specific. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
Edward Chan <[EMAIL PROTECTED]> wrote: > Thanks for the reply. So I'm a bit confused as to how different crypto > packages interoperate. I've been having a problem with MS CryptoAPI > doing a DH key exchange with OpenSSL. There are some quirks such as > byte order, and a couple other things. But for the most part, I have it > working. But occasionally, I get failures which I suspect is due to the > BIGNUM format of the pubkey that is exchanged. Has anybody had success > getting these 2 libs working together? This might be a BN padding problem. In fact, I had the same problem when doing interoperability tests between xyssl and openssl, sometimes after the EDH key exchange I got a bad record mac (this was however quite rare). The reason was that the pre-master key, basically the shared DH secret was incorrectly padded with 0s (like for RSA), whereas openssl doesn't. Christophe > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Marek Marcola > Sent: Tuesday, April 17, 2007 1:55 AM > To: openssl-users@openssl.org > Subject: RE: BIGNUM library > > Hello, > > Is there a specification on the format of a BIGNUM that someone can > > point me to? > Probably headers files. > > > Is there a standard encoding/format that everyone adheres to? > Probably not. > > > Or would different libraries have their own encodings? I hope not. > Most bignum libraries use their own formats, in general they are > very close (structure with words/limbs, some counters, sign > variable ...). Sometimes APIs between libraries are almost the same. > > > How easy or difficult would it be to extract just the BIGNUM library > > from OpenSSL? Are there any documents on how to do this? > Very ease, but if you think about bignum library I think GMP > is better than extract from OpenSSL. > > Best regards, > -- > Marek Marcola <[EMAIL PROTECTED]> > > __ > OpenSSL Project http://www.openssl.org > User Support Mailing Listopenssl-users@openssl.org > Automated List Manager [EMAIL PROTECTED] > __ > OpenSSL Project http://www.openssl.org > User Support Mailing Listopenssl-users@openssl.org > Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Right, I see what you're saying. I realize I'm not actually passing a BIGNUM. Let me clarify, this is what I'm doing. First I call DH_new() to get a new DH object. Then I fill in the p & g members of the DH obj with pre-generated prime and generator. Then I call DH_generate_key() which generates the pub_key and priv_key pair. I then do a BN_bn2bin() on the pub_key member to get a byte array representation of the BIGNUM. This is then passed to CryptoAPI. What is the binary format returned by BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the time. But occasionally fails. Some times OpenSSL returns me an pubkey < 128 bytes, whereas CryptoAPI seems to always expect 128 (and similarly, always returns me 128 bytes). So in cases where OpenSSL gives me < 128, I front pad with zeros. And this also seems to work most of the time. But very rarely, I'll still get a failure. But even if OpenSSL gives me back 128 bytes, I still sporatically get failures. So I suspect there is some incompatibility with the binary representations of the BIGNUM. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Tuesday, April 17, 2007 8:49 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007 at 08:42:12AM -0700, Edward Chan wrote: > Thanks for the reply. So I'm a bit confused as to how different crypto > packages interoperate. I've been having a problem with MS CryptoAPI > doing a DH key exchange with OpenSSL. There are some quirks such as > byte order, and a couple other things. But for the most part, I have it > working. But occasionally, I get failures which I suspect is due to the > BIGNUM format of the pubkey that is exchanged. Has anybody had success > getting these 2 libs working together? You are confusing BIGNUM which is an internal library structure for performing high precision integer arithmetic, with ASN.1 which is an on the wire format for serialized data. The protocols that exchange public keys, ... are not sending BIGNUM structures around, OpenSSL serializes and de-deserializes BIGNUMs to/from appropriate ASN.1 structures. For example X.509v3 is an ASN.1 data structure that contains some large integers, but they are not in any sense BIGNUMs until OpenSSL parses the ASN.1 and constructs its own (X509 *) structure, in which some of the fields ultimately contain BIGNUM references. The serialized ASN.1 is standard, the in-memory (X509 *) is OpenSSL specific. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Tue, Apr 17, 2007, Edward Chan wrote: > Right, I see what you're saying. I realize I'm not actually passing a > BIGNUM. Let me clarify, this is what I'm doing. > > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. > A format difference would be unlikely to cause problems only occasionally. BN_bn2bin() outputs in big endian format with no padding ignoring any sign. I suggest you log the key pairs of both sides in the case where you get a failure and the shared secrets from both implementations. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Hello, > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? No. > It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. My proposition is to check the sign of generated pub_key (before bn2bin) Sign should be positive, but ... When using BN_bn2bin() sign is ignored (if negative). Best regards, -- Marek Marcola <[EMAIL PROTECTED]> __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
The problem with CryptoAPI is that it doesn't give you direct access to the shared secret. But I suspect it is wrong since the encryption/decryption fails (I encrypt something, and decrypt it, to make sure it is the same as the original). I agree that a format difference is unlikely. That's why I suspect it may be related to padding. I checked the binary output from BN_bn2bin() and it does seem to be ASN.1 formatted. At least it matches the data from a call to BN_to_ASN1_INTEGER(). But like Dr. Steve said, it is not likely a format difference. The endian-ness is a problem. MS likes it in little-endian. That's why I have to reverse the bytes of the pubkey before passing it to CAPI. So what I do is, BN_bn2bin(dh->pub_key, buf); ReverseBytes(buf, size); // MS CAPI requires this in little-endian MS CAPI also seems to require that the pubkey always be 128 bytes. So here, I pad with zeros before reversing the byte order. This works almost all the time this works. But I do still get very infrequent failures. But more often, I get failures even when OpenSSL is giving me exactly 128 bytes. I think the sign seems to be a good place to investigate. I'll let you know what I find. Thanks! -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dr. Stephen Henson Sent: Tuesday, April 17, 2007 11:03 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007, Edward Chan wrote: > Right, I see what you're saying. I realize I'm not actually passing a > BIGNUM. Let me clarify, this is what I'm doing. > > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. > A format difference would be unlikely to cause problems only occasionally. BN_bn2bin() outputs in big endian format with no padding ignoring any sign. I suggest you log the key pairs of both sides in the case where you get a failure and the shared secrets from both implementations. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Arg, I'm still getting failures even though the pubkey computed by openssl is not negative and padding is not required. Any other ideas? I'm desperate. A year's supply of Guinness for the person who can help here :) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Edward Chan Sent: Tuesday, April 17, 2007 11:15 AM To: openssl-users@openssl.org Subject: RE: BIGNUM library The problem with CryptoAPI is that it doesn't give you direct access to the shared secret. But I suspect it is wrong since the encryption/decryption fails (I encrypt something, and decrypt it, to make sure it is the same as the original). I agree that a format difference is unlikely. That's why I suspect it may be related to padding. I checked the binary output from BN_bn2bin() and it does seem to be ASN.1 formatted. At least it matches the data from a call to BN_to_ASN1_INTEGER(). But like Dr. Steve said, it is not likely a format difference. The endian-ness is a problem. MS likes it in little-endian. That's why I have to reverse the bytes of the pubkey before passing it to CAPI. So what I do is, BN_bn2bin(dh->pub_key, buf); ReverseBytes(buf, size); // MS CAPI requires this in little-endian MS CAPI also seems to require that the pubkey always be 128 bytes. So here, I pad with zeros before reversing the byte order. This works almost all the time this works. But I do still get very infrequent failures. But more often, I get failures even when OpenSSL is giving me exactly 128 bytes. I think the sign seems to be a good place to investigate. I'll let you know what I find. Thanks! -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dr. Stephen Henson Sent: Tuesday, April 17, 2007 11:03 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007, Edward Chan wrote: > Right, I see what you're saying. I realize I'm not actually passing a > BIGNUM. Let me clarify, this is what I'm doing. > > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. > A format difference would be unlikely to cause problems only occasionally. BN_bn2bin() outputs in big endian format with no padding ignoring any sign. I suggest you log the key pairs of both sides in the case where you get a failure and the shared secrets from both implementations. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Tue, Apr 17, 2007 at 11:15:23AM -0700, Edward Chan wrote: > BN_bn2bin(dh->pub_key, buf); > ReverseBytes(buf, size); // MS CAPI requires this in little-endian Reversing the BER encoded bytes of a big endian integer does not yield the BER encoding of the correspoding little-endian integer. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Hmm, that's interesting. Then why does this seem to work most of the time? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Tuesday, April 17, 2007 11:32 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007 at 11:15:23AM -0700, Edward Chan wrote: > BN_bn2bin(dh->pub_key, buf); > ReverseBytes(buf, size); // MS CAPI requires this in little-endian Reversing the BER encoded bytes of a big endian integer does not yield the BER encoding of the correspoding little-endian integer. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Tue, Apr 17, 2007 at 02:31:50PM -0400, Victor Duchovni wrote: > On Tue, Apr 17, 2007 at 11:15:23AM -0700, Edward Chan wrote: > > > BN_bn2bin(dh->pub_key, buf); > > ReverseBytes(buf, size); // MS CAPI requires this in little-endian > > Reversing the BER encoded bytes of a big endian integer does not yield > the BER encoding of the correspoding little-endian integer. But of course BN_bn2bin() does not return BER encoded data (ASN.1), so the issue is elsewhere. How does CryptoAPI handle signs of little-endian numbers? Are they signed or unsigned? -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Honestly, I have no idea how CryptoAPI handles this stuff. And documentation is hard to come by. But I can say that in the failure cases I'm seeing, OpenSSL is indeed giving me back a positive BIGNUM. So at least on our side it doesn't not appear to be a signed/unsigned issue. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Tuesday, April 17, 2007 11:38 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007 at 02:31:50PM -0400, Victor Duchovni wrote: > On Tue, Apr 17, 2007 at 11:15:23AM -0700, Edward Chan wrote: > > > BN_bn2bin(dh->pub_key, buf); > > ReverseBytes(buf, size); // MS CAPI requires this in little-endian > > Reversing the BER encoded bytes of a big endian integer does not yield > the BER encoding of the correspoding little-endian integer. But of course BN_bn2bin() does not return BER encoded data (ASN.1), so the issue is elsewhere. How does CryptoAPI handle signs of little-endian numbers? Are they signed or unsigned? -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Tue, Apr 17, 2007, Edward Chan wrote: > The problem with CryptoAPI is that it doesn't give you direct access to > the shared secret. But I suspect it is wrong since the > encryption/decryption fails (I encrypt something, and decrypt it, to > make sure it is the same as the original). > It doesn't give you *direct* access to the shared secret or indeed other types of symmetric or asymmetric keys but there are back door ways of getting hold of the key anyway. One way is to encrypt the key using a public key to which you know the corresponsing private key and then obtaining the unencrypted result using OpenSSL. Another trick is in an MS KB article somewhere which relies on using a key with an exponent of 1. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
I was talking to the MS support guy who wrote that article. He said he has spoken with the engineers and assures me that it is not possible with DH keys. But if someone knows otherwise, I'd really love some sample code. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dr. Stephen Henson Sent: Tuesday, April 17, 2007 4:40 PM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Tue, Apr 17, 2007, Edward Chan wrote: > The problem with CryptoAPI is that it doesn't give you direct access to > the shared secret. But I suspect it is wrong since the > encryption/decryption fails (I encrypt something, and decrypt it, to > make sure it is the same as the original). > It doesn't give you *direct* access to the shared secret or indeed other types of symmetric or asymmetric keys but there are back door ways of getting hold of the key anyway. One way is to encrypt the key using a public key to which you know the corresponsing private key and then obtaining the unencrypted result using OpenSSL. Another trick is in an MS KB article somewhere which relies on using a key with an exponent of 1. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Wed, Apr 18, 2007, Edward Chan wrote: > I was talking to the MS support guy who wrote that article. He said he > has spoken with the engineers and assures me that it is not possible > with DH keys. > > But if someone knows otherwise, I'd really love some sample code. > Well the alternative of using a known key still exists: either a symmetric or asymmetric one. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
I have more info on this now. I just tried openssl with bsafe crypto-c me. Again, I get sporadic failures. When I compare the 2 computed shared secrets, I see that they are actually the same, except that bsafe has some zero padded bytes at the beginning, even though it says it computed 128 bytes. So for example, openssl[0, 127] == bsafe[1, 128], where bsafe[0] == 0 or openssl[0, 127] == bsafe[2, 129], where bsafe[0] == bsafe[1] == 0 Anybody have any ideas. I haven't tried MS CAPI to BSAFE yet, but I have a feeling these 2 libs may work together. Can somebody point me to a spec on the ASN.1 format for BIGNUM's over the wire? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dr. Stephen Henson Sent: Thursday, April 19, 2007 3:14 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Wed, Apr 18, 2007, Edward Chan wrote: > I was talking to the MS support guy who wrote that article. He said he > has spoken with the engineers and assures me that it is not possible > with DH keys. > > But if someone knows otherwise, I'd really love some sample code. > Well the alternative of using a known key still exists: either a symmetric or asymmetric one. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
Edward Chan wrote: > I have more info on this now. I just tried openssl with bsafe crypto-c > me. Again, I get sporadic failures. When I compare the 2 computed > shared secrets, I see that they are actually the same, except that bsafe > has some zero padded bytes at the beginning, even though it says it > computed 128 bytes. > > So for example, > > openssl[0, 127] == bsafe[1, 128], where bsafe[0] == 0 > or > openssl[0, 127] == bsafe[2, 129], where bsafe[0] == bsafe[1] == 0 > > Anybody have any ideas. I haven't tried MS CAPI to BSAFE yet, but I > have a feeling these 2 libs may work together. > > Can somebody point me to a spec on the ASN.1 format for BIGNUM's over > the wire? > Well the case with bsafe[0]==bsafe[1]==0 definitely cannot be ASN.1 integer compliant, because ASN.1 DER specifies that you must use the shortest possible encoding for a number. Having two leading 0s isn't definitely shortest (or unique), since you can get the same number with just a single leading 0. In case you want the spec for ASN.1 bignum, it should be the X.691 for Integer encoding. (i'm assuming DER here) -jb -- The biggest problem with communication is the illusion that it has occurred. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Fri, Apr 20, 2007 at 01:12:29PM +0530, jimmy wrote: > Edward Chan wrote: > > I have more info on this now. I just tried openssl with bsafe crypto-c > > me. Again, I get sporadic failures. When I compare the 2 computed > > shared secrets, I see that they are actually the same, except that bsafe > > has some zero padded bytes at the beginning, even though it says it > > computed 128 bytes. > > > > So for example, > > > > openssl[0, 127] == bsafe[1, 128], where bsafe[0] == 0 > > or > > openssl[0, 127] == bsafe[2, 129], where bsafe[0] == bsafe[1] == 0 > > > > Anybody have any ideas. I haven't tried MS CAPI to BSAFE yet, but I > > have a feeling these 2 libs may work together. > > > > Can somebody point me to a spec on the ASN.1 format for BIGNUM's over > > the wire? > > > > Well the case with bsafe[0]==bsafe[1]==0 definitely cannot be ASN.1 > integer compliant, because ASN.1 DER specifies that you must use the > shortest possible encoding for a number. > > Having two leading 0s isn't definitely shortest (or unique), since you > can get the same number with just a single leading 0. > > In case you want the spec for ASN.1 bignum, it should be the X.691 for > Integer encoding. (i'm assuming DER here) Is an issue with on-the-wire interoperability of a standardized protocol, or an issue with out of band migration of key material between different systems? If the latter, there are no standards. The OP is still being fast and loose with the distinction between the private formats used internally by applications and libraries, and the public formats used on the wire... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
I think the issue is with on the wire interoperability. Let me summarize my tests so far. Openssl to openssl - this works 100% CAP to CAPI - works 100% OpenSSL to CAPI - sporadic failures XySSL to CAPI - sporadic failures OpenSSL to XySSL - works 100% Because CAPI doesn't give you access to the computed shared secret, I can't say for sure that the computed shared secret is different from openssl/xyssl. But I'm 99% sure this is the reason for the failure. And because I got another crypto lib to work with openssl, which also failed with CAPI in the same manner, I assumed the problem lie with CAPI. But now, I just tried another commercial crypto lib, RSA's BSAFE Crypto-C ME. OpenSSL to BSAFE - sporadic failures BSAFE to CAPI - works 100% What are the chances that 2 commercial crypto libraries from heavy weights Microsoft and RSA have similar bugs? Maybe the chances are high :) But at this point, I'm starting to think, dare I say it, that there might possibly be a bug in OpenSSL? Anybody else have ideas? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Friday, April 20, 2007 7:53 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Fri, Apr 20, 2007 at 01:12:29PM +0530, jimmy wrote: > Edward Chan wrote: > > I have more info on this now. I just tried openssl with bsafe crypto-c > > me. Again, I get sporadic failures. When I compare the 2 computed > > shared secrets, I see that they are actually the same, except that bsafe > > has some zero padded bytes at the beginning, even though it says it > > computed 128 bytes. > > > > So for example, > > > > openssl[0, 127] == bsafe[1, 128], where bsafe[0] == 0 > > or > > openssl[0, 127] == bsafe[2, 129], where bsafe[0] == bsafe[1] == 0 > > > > Anybody have any ideas. I haven't tried MS CAPI to BSAFE yet, but I > > have a feeling these 2 libs may work together. > > > > Can somebody point me to a spec on the ASN.1 format for BIGNUM's over > > the wire? > > > > Well the case with bsafe[0]==bsafe[1]==0 definitely cannot be ASN.1 > integer compliant, because ASN.1 DER specifies that you must use the > shortest possible encoding for a number. > > Having two leading 0s isn't definitely shortest (or unique), since you > can get the same number with just a single leading 0. > > In case you want the spec for ASN.1 bignum, it should be the X.691 for > Integer encoding. (i'm assuming DER here) Is an issue with on-the-wire interoperability of a standardized protocol, or an issue with out of band migration of key material between different systems? If the latter, there are no standards. The OP is still being fast and loose with the distinction between the private formats used internally by applications and libraries, and the public formats used on the wire... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Fri, Apr 20, 2007 at 10:34:55AM -0700, Edward Chan wrote: > I think the issue is with on the wire interoperability. Let me > summarize my tests so far. > > Openssl to openssl - this works 100% > CAP to CAPI - works 100% > OpenSSL to CAPI - sporadic failures > XySSL to CAPI - sporadic failures > OpenSSL to XySSL - works 100% What works? What protocol are you using, and what software is constructing the packets that go on the wire? If you are building your own protocol over the multiple low-level crypto libraries, and doing your own data serialization, the you are responsible for ensuring on the wire compatibility of the of serialized data. I think you need to better explain what problem you are really solving... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
Edward Chan <[EMAIL PROTECTED]> wrote: > Openssl to openssl - this works 100% > CAP to CAPI - works 100% > OpenSSL to CAPI - sporadic failures > XySSL to CAPI - sporadic failures > OpenSSL to XySSL - works 100% I noted earlier that when your testcode failed, OpenSSL's secret always begins with 0xCC; however sometimes the test passed with the first byte of the secret equal to 0xCC. It just realized that, with VS uninitialized memory is filled with 0xCC. So the error was not in OpenSSL or CAPI, but rather in the incorrect use of the function DH_generate_key (the return value was not taken into account properly when setting up the shared secret). A simple fix is to add zero padding when the secret is less than 128 bytes. > Because CAPI doesn't give you access to the computed shared secret, I > can't say for sure that the computed shared secret is different from > openssl/xyssl. But I'm 99% sure this is the reason for the failure. Even though MS says otherwise, it is possible to access the DH shared secret computed by CAPI, with a bit of trickery. When disassembling dssenh.dll using the associated debug symbols, it appears that the secret is internally computed by UseDHKey, which calls DHExpo, which calls BN_mod_exp, which itself is a wrapper for _mod_exp(). Then the function BN_from_modular is called, and the result (stored in the second argument of _BN_from_modular) is the shared secret. .text:68106ED2 pushebx .text:68106ED3 lea eax, [ebp+var_200] ; shared secret .text:68106ED9 pusheax .text:68106EDA lea eax, [ebp+var_800] .text:68106EE0 pusheax .text:68106EE1 call[EMAIL PROTECTED] .text:68106EE6 testeax, eax Setting up a breakpoint at 0x68106EE6, one can therefore obtain this value directly by looking it up in the stack. > What are the chances that 2 commercial crypto libraries from heavy > weights Microsoft and RSA have similar bugs? Maybe the chances are high > :) But at this point, I'm starting to think, dare I say it, that there > might possibly be a bug in OpenSSL? Anybody else have ideas? In fact I had the same bug wrt/ the DH code in XySSL a couple months ago, not taking into account the fact that OpenSSL does not add zero padding. Christophe __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
> So the error was not in OpenSSL or CAPI, but rather in the incorrect > use of the function DH_generate_key (the return value was not taken > into account properly when setting up the shared secret). A simple > fix is to add zero padding when the secret is less than 128 bytes. My mistake, it's the function DH_compute_key(). Christophe __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
But I think this always returned me 128 bytes. So am I supposed to bzero the output buffer first? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Christophe Devine Sent: Friday, April 20, 2007 12:00 PM To: openssl-users@openssl.org Subject: Re: BIGNUM library > So the error was not in OpenSSL or CAPI, but rather in the incorrect > use of the function DH_generate_key (the return value was not taken > into account properly when setting up the shared secret). A simple > fix is to add zero padding when the secret is less than 128 bytes. My mistake, it's the function DH_compute_key(). Christophe __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
I apologize for the confusion. I thought I had stated the problem before. I'm basically trying to do a DH key exchange between different crypto libraries. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Friday, April 20, 2007 11:00 AM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Fri, Apr 20, 2007 at 10:34:55AM -0700, Edward Chan wrote: > I think the issue is with on the wire interoperability. Let me > summarize my tests so far. > > Openssl to openssl - this works 100% > CAP to CAPI - works 100% > OpenSSL to CAPI - sporadic failures > XySSL to CAPI - sporadic failures > OpenSSL to XySSL - works 100% What works? What protocol are you using, and what software is constructing the packets that go on the wire? If you are building your own protocol over the multiple low-level crypto libraries, and doing your own data serialization, the you are responsible for ensuring on the wire compatibility of the of serialized data. I think you need to better explain what problem you are really solving... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Fri, Apr 20, 2007 at 03:43:41PM -0700, Edward Chan wrote: > I apologize for the confusion. I thought I had stated the problem > before. I'm basically trying to do a DH key exchange between different > crypto libraries. Why an explicit DH key exchange and not TLS, which is interoperable, and authenticates the DH exchange, ... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
Edward Chan <[EMAIL PROTECTED]> wrote:
> But I think this always returned me 128 bytes. So am I supposed to
> bzero the output buffer first?
Here's how I fixed the bug (not very elegant, it was a quick hack)
int i, ret = DH_compute_key(secret, pkey, m_dh);
if( ret > 0 && ret < 128 )
{
for(i = ret; i >= 0; i--)
secret[i+1] = secret[i];
memset(secret, 0, 128 - ret);
}
ReverseBytes(secret, size);
Christophe
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
I thought I tried this, but let me try again.
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Christophe Devine
Sent: Saturday, April 21, 2007 2:03 AM
To: openssl-users@openssl.org
Subject: Re: BIGNUM library
Edward Chan <[EMAIL PROTECTED]> wrote:
> But I think this always returned me 128 bytes. So am I supposed to
> bzero the output buffer first?
Here's how I fixed the bug (not very elegant, it was a quick hack)
int i, ret = DH_compute_key(secret, pkey, m_dh);
if( ret > 0 && ret < 128 )
{
for(i = ret; i >= 0; i--)
secret[i+1] = secret[i];
memset(secret, 0, 128 - ret);
}
ReverseBytes(secret, size);
Christophe
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Excuse my ignorance, but isn't TLS a protocol, whereas DH an algorithm? Even if TLS is used, if it does a DH key exchange, I think I would still have the same interop problems. I'm basically testing interop of different crypto libs. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Friday, April 20, 2007 9:31 PM To: openssl-users@openssl.org Subject: Re: BIGNUM library On Fri, Apr 20, 2007 at 03:43:41PM -0700, Edward Chan wrote: > I apologize for the confusion. I thought I had stated the problem > before. I'm basically trying to do a DH key exchange between different > crypto libraries. Why an explicit DH key exchange and not TLS, which is interoperable, and authenticates the DH exchange, ... -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED] __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
Re: BIGNUM library
On Sat, Apr 21, 2007 at 07:45:24AM -0700, Edward Chan wrote: > Excuse my ignorance, but isn't TLS a protocol, whereas DH an algorithm? Exactly, so the protocol has known on-the-wire characteristics, but the algorithm is a building block, and you have to build the protocol yourself, and in some rare cases this is justified, but in most cases one is better off using a fielded protocol. I am curious why you need to build your own protocol... > Even if TLS is used, if it does a DH key exchange, I think I would still > have the same interop problems. But it does use DH, and does not have the problem you observe. > I'm basically testing interop of different crypto libs. You are not testing library inter-operability, you are testing inter-operability of your glue code, which is wrapped in a non interoperable way, around multiple implementations of the same algorithm, that all yield the same logical result, but use different internal result layouts. To test library interoperability, test TLS or S/MIME, which are *protocols* implemented by the libraries. -- Viktor. __ OpenSSL Project http://www.openssl.org User Support Mailing Listopenssl-users@openssl.org Automated List Manager [EMAIL PROTECTED]
RE: BIGNUM library
Christophe, you're right. I just looked at my coe again, and I was not
checking the return value of DH_compute_key() for the size of the
computed shared secret; I was assuming it to be the same size as that
returned by DH_size(), which is obviously not a valid assumption. All
the other libs return the size as an in/out arg where on the way in, the
arg is set to the size of the output buffer used to store the secret,
and on the way out it is set to the size of the secret. So I had
overlooked that the size was actually returned via the return value.
Thanks for your help on this!
-Original Message-
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of Christophe Devine
Sent: Saturday, April 21, 2007 2:03 AM
To: openssl-users@openssl.org
Subject: Re: BIGNUM library
Edward Chan <[EMAIL PROTECTED]> wrote:
> But I think this always returned me 128 bytes. So am I supposed to
> bzero the output buffer first?
Here's how I fixed the bug (not very elegant, it was a quick hack)
int i, ret = DH_compute_key(secret, pkey, m_dh);
if( ret > 0 && ret < 128 )
{
for(i = ret; i >= 0; i--)
secret[i+1] = secret[i];
memset(secret, 0, 128 - ret);
}
ReverseBytes(secret, size);
Christophe
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
__
OpenSSL Project http://www.openssl.org
User Support Mailing Listopenssl-users@openssl.org
Automated List Manager [EMAIL PROTECTED]
