Chris Angelico <ros...@gmail.com> Wrote in message: > On Thu, Sep 25, 2014 at 5:56 AM, Dave Angel <da...@davea.name> wrote: >> Your definition is not nearly as concrete as you think. Is the >> first number considered to be exact, and we'll only check the >> second? Will the factor always be an int, and thus >> exact? > > Apologies: the definition is concrete, just under-specified in the post. > > No, both numbers are inexact; most notably, one third of the factor > might be involved - both of these should be correct: > > ["1.333", "40"] > ["1.33", "40"] > > But if it's easier to assume that one or t'other must be exact, that's > fine too - I can just run the check twice, once each way. >
You could assume that the one with more significant digits is the exact one. Another approach is to see if twiddling the last digit gets you from too high to too low. I used this approach to check a trig package I wrote in '75. An answer was close enough if twiddling the last digit changed the result value from too high to too low, or if twiddling the last digit of the argument did. I needed to go both ways because the curve sometimes had a slope greater than one. Oh, and don't forget to set the precision of Decimal to a couple more than the sum of the digits. -- DaveA -- https://mail.python.org/mailman/listinfo/python-list
