On Tue, Jan 13, 2015 at 07:03:17AM +0100, John Darrington wrote: > This change adds a small module to perform floating point arithmetic > with a decimal base, and uses it to calculate the graticule marks for > charts. > > Rationale: Graticule marks want to be displayed in decimal. However > not all decimal values can be represented in a double precision > floating point binary. This approach pushes the precision loss from > the value of the mark, to the position on the chart - we don't > particularly care if a tick mark is a fraction of a pixel out of place, > but we do care if 4.0 is displayed as 3.9999999999
Can you say a few words about the assumptions? At first glance, some of the functions seem to suffer from catastrophic loss of precision upon integer overflow, e.g. doesn't that happen in decimal_multiply() if the intermediate result is greater than ORD_MAX? (Is some kind of care taken to make sure that this cannot happen?) _______________________________________________ pspp-dev mailing list [email protected] https://lists.gnu.org/mailman/listinfo/pspp-dev
