Right, I see what you're saying. I realize I'm not actually passing a BIGNUM. Let me clarify, this is what I'm doing.
First I call DH_new() to get a new DH object. Then I fill in the p & g members of the DH obj with pre-generated prime and generator. Then I call DH_generate_key() which generates the pub_key and priv_key pair. I then do a BN_bn2bin() on the pub_key member to get a byte array representation of the BIGNUM. This is then passed to CryptoAPI. What is the binary format returned by BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the time. But occasionally fails. Some times OpenSSL returns me an pubkey < 128 bytes, whereas CryptoAPI seems to always expect 128 (and similarly, always returns me 128 bytes). So in cases where OpenSSL gives me < 128, I front pad with zeros. And this also seems to work most of the time. But very rarely, I'll still get a failure. But even if OpenSSL gives me back 128 bytes, I still sporatically get failures. So I suspect there is some incompatibility with the binary representations of the BIGNUM. -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Victor Duchovni Sent: Tuesday, April 17, 2007 8:49 AM To: [email protected] Subject: Re: BIGNUM library On Tue, Apr 17, 2007 at 08:42:12AM -0700, Edward Chan wrote: > Thanks for the reply. So I'm a bit confused as to how different crypto > packages interoperate. I've been having a problem with MS CryptoAPI > doing a DH key exchange with OpenSSL. There are some quirks such as > byte order, and a couple other things. But for the most part, I have it > working. But occasionally, I get failures which I suspect is due to the > BIGNUM format of the pubkey that is exchanged. Has anybody had success > getting these 2 libs working together? You are confusing BIGNUM which is an internal library structure for performing high precision integer arithmetic, with ASN.1 which is an on the wire format for serialized data. The protocols that exchange public keys, ... are not sending BIGNUM structures around, OpenSSL serializes and de-deserializes BIGNUMs to/from appropriate ASN.1 structures. For example X.509v3 is an ASN.1 data structure that contains some large integers, but they are not in any sense BIGNUMs until OpenSSL parses the ASN.1 and constructs its own (X509 *) structure, in which some of the fields ultimately contain BIGNUM references. The serialized ASN.1 is standard, the in-memory (X509 *) is OpenSSL specific. -- Viktor. ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [email protected] Automated List Manager [EMAIL PROTECTED] ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [email protected] Automated List Manager [EMAIL PROTECTED]
