Hello, > First I call DH_new() to get a new DH object. > > Then I fill in the p & g members of the DH obj with pre-generated prime > and generator. > > Then I call DH_generate_key() which generates the pub_key and priv_key > pair. > > I then do a BN_bn2bin() on the pub_key member to get a byte array > representation of the BIGNUM. > > This is then passed to CryptoAPI. What is the binary format returned by > BN_bn2bin()? Is this ASN.1 formatted? No.
> It seems to work most of the > time. But occasionally fails. Some times OpenSSL returns me an pubkey > < 128 bytes, whereas CryptoAPI seems to always expect 128 (and > similarly, always returns me 128 bytes). So in cases where OpenSSL > gives me < 128, I front pad with zeros. And this also seems to work > most of the time. But very rarely, I'll still get a failure. But even > if OpenSSL gives me back 128 bytes, I still sporatically get failures. > So I suspect there is some incompatibility with the binary > representations of the BIGNUM. My proposition is to check the sign of generated pub_key (before bn2bin) Sign should be positive, but ... When using BN_bn2bin() sign is ignored (if negative). Best regards, -- Marek Marcola <[EMAIL PROTECTED]> ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List [email protected] Automated List Manager [EMAIL PROTECTED]
