On Wed, 21 Apr 2010 02:58:06 am Lowell Tackett wrote: > I'm running headlong into the dilemma of binary math representation, with game-ending consequences, e.g.: > >>> 0.15 > > 0.14999999999999999 > > Obviously, any attempts to manipulate this value, under the misguided > assumption that it is truly "0.15" are ill-advised, with inevitable > bad results.
That really depends on what sort of manipulation you are doing. >>> x = 0.15 >>> x 0.14999999999999999 >>> x*100 == 15 True Seems pretty accurate to me. However: >>> 18.15*100 == 1815 False The simplest, roughest way to fix these sorts of problems (at the risk of creating *other* problems!) is to hit them with a hammer: >>> round(18.15*100) == 1815 True [...] > What I'm shooting for, by the way, is an algorithm that converts a > deg/min/sec formatted number to decimal degrees. It [mostly] worked, > until I stumbled upon the peculiar cases of 15 minutes and/or 45 > minutes, which exposed the flaw. I'm afraid that due to the nature of floating point, this is a hard problem. Even the professionals at Hewlett-Packard's scientific calculator division don't always get it right, and they are *extremely* careful: http://www.hpmuseum.org/cgi-sys/cgiwrap/hpmuseum/archv018.cgi?read=132690 The best result I can suggest is, change the problem! Don't pass degrees-minutes-seconds around using a floating point value, but as a tuple with distinct (DEG, MIN, SEC) integer values. Or create a custom class. But if you really need D.MMSS floats, then something like this should be a good start.: def dms2deg(f): """Convert a floating point number formatted as D.MMSS into degrees. """ mmss, d = math.modf(f) assert d == int(f) if mmss >= 0.60: raise ValueError( 'bad fractional part, expected < .60 but got %f' % mmss) mmss *= 100 m = round(mmss) if m >= 60: raise ValueError('bad minutes, expected < 60 but got %d' % m) s = round((mmss - m)*100, 8) if not 0 <= s < 60.0: raise ValueError('bad seconds, expected < 60.0 but got %f' % s) return d + m/60.0 + s/3600.0 >>> dms2deg(18.15) 18.25 >>> dms2deg(18.1515) 18.254166666666666 which compares well to my HP-48GX: 18.15 HMS-> gives 18.25, and: 18.1515 HMS-> gives 18.2541666667. Note though that this still fails with some valid input. I will leave fixing it as an exercise (or I might work on it later, time permitting). -- Steven D'Aprano _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor