On Fri, 27 Oct 2006 07:57:10 -0500, Bill Hart <[EMAIL PROTECTED]> wrote: > William Stein wrote: > >> Literals are parsed by the RealNumber command, which you can set to >> whatever >> you want (as I explained a day or two ago on sage-devel). > > Whoops, I see that now. I can't imagine how I managed to miss that. It > works, thanks. > > By the way, I noted above that one more decimal place was being > displayed for floats of the "default" precision of 53 bits than should > be. This was slightly incorrect. > > This was also based on the assumption that the sign is not stored in > that 53 bits, but I think it actually is stored there. Thus, here is my > revised analysis: > > The binary value for a precision of 53 bits including a sign is correct > to one part in 2^52. We want that to be less than the smallest decimal > to be displayed. Now 1/2^52 = 2.22E-16. Thus for a decimal mantissa of > 9.9999999... one wants the last digit displayed to be bigger than > 10*2.22E^-16 = 2.22E-15, i.e. the last decimal digit of the mantissa > displayed should be 1E-14, which implies there should be 15 significant > decimal digits displayed. There are currently 17 displayed. Thus this > needs to be reduced by 2. > > This example should give a suitable formula for any binary precision.
I like this. I've implemented it. I'm sure it will avoid a lot of confusion in the future. William --~--~---------~--~----~------------~-------~--~----~ To post to this group, send email to sage-devel@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sage-devel URLs: http://sage.scipy.org/sage/ and http://modular.math.washington.edu/sage/ -~----------~----~----~----~------~----~------~--~---
