For VAX/VMS people: I think I found a solution. Shifting down the dividend down one more step seems to be it (or at least it avoids getting the temporary quotient 0x80000000, which is what is needed to get EDIV to accept the result as valid). However, it also means having a 33 bit shifted up quotient for a moment, until the last part of the algorithm does it's arithmetics on the (in all tests I've done) too large remainder (and therefore the quotient as well).
I'll do some tests, and commit as soon as this is verified. [[EMAIL PROTECTED] - Fri Dec 20 16:46:22 2002]: > The problem is when an EDIV will produce a negative quotient from the > division of two positive numbers. An example is the following call: > > q = bn_div_words(0xC0171716, 0xFFFFAB9A, 0xC0171717); > > q is supposed to get the result 0xFFFFFFFF. Unfortunately, the way > the numbers are handled to avoid negative numbers (I've been bending > backwards to get that to work), EDIV is asked to perform the following > operation: > > 3005C5C5BFFFEAE6 / 600B8B8B > > The result of such an operation is 80000000, which is a negative > number according to EDIV, and since that's the result of dividing two > positive numbers, this is an integer overflow, and the result becomes > incorrect. > > For the arguments (n0, n1, d) to bn_div_words(), I've found out that > the problem occurs under the following condition: > > ((n0 & 0x80000000) && (d & 0x80000000) > && (d >> 2) <= (n0 >> 2) && (d >= 0x80000000)) > > I'm trying to find a method to handle this case, currently involving > subtracting 0x80000000 from n0 and trying to adjust the result > accordingly, but I haven't come very far. > > This is a showstopper on VAX. -- Richard Levitte ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [EMAIL PROTECTED] Automated List Manager [EMAIL PROTECTED]
