I don't have an answer, other than to say there has been a LOT of
discussion over the years about the limits of number precision in
LiveCode and a number of enhancement requests and 3rd party libraries
that have attempted to provide a framework for high precision math. You
might search the list archives or search the forum on the LC web site
for past discussions about number precision.
On 3/20/2021 3:06 PM, jbv via use-livecode wrote:
Hi list,
I am a bit lost :
set numberformat to ".#########################"
put -0.0000007347880794884118999 / 1000
-- returns -0.0000000007347880794884119 which is ok
put -0.0000007347880794884118999 / 1000000
-- returns -0.0000000000007347880794884 -> also ok
put -0.0000007347880794884118999 / 100000000
-- returns -0.0000000000000073478807949 -> still ok
put -0.0000007347880794884118999 / 1000000000
-- returns 0 -> ?????
Any explanations ?
Thanks in advance.
Best,
jbv
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your
subscription preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
