The problem is when an EDIV will produce a negative quotient from the
division of two positive numbers. An example is the following call:
q = bn_div_words(0xC0171716, 0xFFFFAB9A, 0xC0171717);
q is supposed to get the result 0xFFFFFFFF. Unfortunately, the way
the numbers are handled to avoid negative numbers (I've been bending
backwards to get that to work), EDIV is asked to perform the following
operation:
3005C5C5BFFFEAE6 / 600B8B8B
The result of such an operation is 80000000, which is a negative
number according to EDIV, and since that's the result of dividing two
positive numbers, this is an integer overflow, and the result becomes
incorrect.
For the arguments (n0, n1, d) to bn_div_words(), I've found out that
the problem occurs under the following condition:
((n0 & 0x80000000) && (d & 0x80000000)
&& (d >> 2) <= (n0 >> 2) && (d >= 0x80000000))
I'm trying to find a method to handle this case, currently involving
subtracting 0x80000000 from n0 and trying to adjust the result
accordingly, but I haven't come very far.
This is a showstopper on VAX.
--
Richard Levitte \ Spannvägen 38, II \ [EMAIL PROTECTED]
Redakteur@Stacken \ S-168 35 BROMMA \ T: +46-8-26 52 47
\ SWEDEN \ or +46-708-26 53 44
Procurator Odiosus Ex Infernis -- [EMAIL PROTECTED]
Member of the OpenSSL development team: http://www.openssl.org/
Unsolicited commercial email is subject to an archival fee of $400.
See <http://www.stacken.kth.se/~levitte/mail/> for more info.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List [EMAIL PROTECTED]
Automated List Manager [EMAIL PROTECTED]
