When built with --with-long-double, I get: 0.707106781186547524436104145140 Only the first 19 decimals match WolframAlpha result. I'm guessing that this is why this is an experimental feature.
Le mardi 17 juillet 2012 à 11:37 +0200, Jean Brefort a écrit : > It's a precision issue. Gnumeric uses C double to store real numbers and > you can't expect more that 17 decimals. Compiling with > --with-long-double would give a better precision, but this has always > been marked as experimental. > > Regards, > Jean > > Le lundi 16 juillet 2012 à 19:24 -0400, Clay Lawrence a écrit : > > Can someone else try this and see if they get the same results I do. > > I'm using Gnumeric Spreadsheet 1.10.16 > > > > > > Format a cell to 30 decimal places and enter =1/sqrt(2) > > I get 0.707106781186547460000000000000 which rounds after 17 places > > instead of 30. > > > > > > I also went to WolframAlpha and calculated 1/√2 to 30+ places; > > 0.7071067811865475244008443621048490392848359376884740 but after > > keying this in directly Gnumeric still rounds it to > > 0.707106781186547570000000000000 as soon as I hit the enter key. > > > > > > Is there a setting somewhere that I can set to get Gnumeric to use all > > of the 30 of the places it displays instead of rounding to 17 places > > and padding the remainder with zeroes? > > > > > > -- > > > > Clay > > > > "There are only 10 kinds of people in the world -- > > Those who understand binary, and those who don't." > > > > > > > > _______________________________________________ > > gnumeric-list mailing list > > gnumeric-list@gnome.org > > https://mail.gnome.org/mailman/listinfo/gnumeric-list > > > _______________________________________________ > gnumeric-list mailing list > gnumeric-list@gnome.org > https://mail.gnome.org/mailman/listinfo/gnumeric-list _______________________________________________ gnumeric-list mailing list gnumeric-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnumeric-list
