17 digits sounds like floating point precision issue: 64 bit IEEE floating point numbers offer 53 bits of integer precision. Converting that to decimal precision, I get:
10 ^. 2^ 53 15.9546 So 15 digits is probably reliable, 16 digits is iffy, and 17 digits is luck. Or, numerically: x: <.o. 10x^20 NB. floating point intermediate result 314159265358979334144 <.@o. 10x^20 NB. integer calculations (special code) 314159265358979323846 -- Raul On Mon, Mar 26, 2012 at 1:35 PM, Dan T. Abell <dab...@txcorp.com> wrote: > But the disagreement is after just the 17th digit. > > 3141592653589793238462643383279 <-- pi > 3141592653589793216413703340032 <-- some other number > agree to here | > > > On 26 Mar 2012, at 10:36, Devon McCormick wrote: > >> Because your original function returns a number and "f" returns a >> literal representation of a number. >> >> On Sun, Mar 25, 2012 at 12:23 PM, Linda Alvord <lindaalv...@verizon.net> >> wrote: >>> Here are some odd things I found so far. >> ... >>> f=: 13 :'":x:<.o.10x^y' >>> f 30 >>> 3141592653589793216413703340032 >>> #f 30 >>> 31 >>> >>> (<.@o.10x^30),<.@o.10x^30 >>> 3141592653589793238462643383279 3141592653589793238462643383279 >>> (f 30),f 30 >>> 31415926535897932164137033400323141592653589793216413703340032 >>> >>> >>> >>> >>> Why don't the original statement and my function f agree? Which is >>> correct? >> ... >> -- >> Devon McCormick, CFA >> ^me^ at acm. >> org is my >> preferred e-mail >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm > > -- > Dan T. Abell :: dabell at txcorp dot com :: 303.444.2452 > Tech-X Corp., 5621 Arapahoe Ave, Ste A, Boulder CO 80303 > http://www.txcorp.com :: 303.748.6894/c 303.448.7756/fx > > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
