At 10:22 -0600 2009/08/07, DIETER ENSSLEN wrote:
>thanks,
>
>i thought i was going to be stuck with 6 or even at times only 5
>significant figures!
>
>hope there are lots more, say at least the 15 that windows excel gives
Normal IEEE 64 bit floating point is how j stores and operates on
floats (I imagine does Excel too). Display precision (formattting) is
a different issue in both J and Excel.
Of course, there are the pitfalls of "funny numbers" because of
floating point representation - e.g.
0j32 ": 0.1, %3
0.10000000000000000555111512312578 0.33333333333333331482961625624739
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm
