On 21/08/17 22:15, Ralf Quint wrote:
On 8/21/2017 3:02 PM, James Richters wrote:> I am having an issue with a simple floating point application. I am setting a variable to a specific value and immediately after I set it, it is not exactly what I set it to. Here's an example>> Draw_GX_Min:=999.999;> Writeln(Draw_GX_Min:3:30);>> The writeln results in 999.999000000000020000000000000000 >> Why is it not 999.999000000000000000000000000000 where did 0.00000000000002 come from?>Out of thin air... Well, kind of. Double floating point means 16 digitsof precision, so when you force a 30 digit precision output, anythingpast 16 digits is random garbage, at best...
And in any event, that's probably much more precision than the GPU is using to generate the final image :-)
-- Mark Morgan Lloyd markMLl .AT. telemetry.co .DOT. uk [Opinions above are the author's, not those of his employers or colleagues] _______________________________________________ fpc-pascal maillist - fpc-pascal@lists.freepascal.org http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal
