On Tue, Apr 17, 2007, Edward Chan wrote: > Right, I see what you're saying. I realize I'm not actually passing a > BIGNUM. Let me clarify, this is what I'm doing. > > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. >
A format difference would be unlikely to cause problems only occasionally. BN_bn2bin() outputs in big endian format with no padding ignoring any sign. I suggest you log the key pairs of both sides in the case where you get a failure and the shared secrets from both implementations. Steve. -- Dr Stephen N. Henson. Email, S/MIME and PGP keys: see homepage OpenSSL project core developer and freelance consultant. Funding needed! Details on homepage. Homepage: http://www.drh-consultancy.demon.co.uk ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [email protected] Automated List Manager [EMAIL PROTECTED]
