Hi Regina, On Wednesday, 2008-06-25 23:42:16 +0200, Regina Henschel wrote:
> I'm working on BETADIST. For certain inputs my version of that function > returns a value, which is near to 1. The formula in the cell is > =BETADIST(B$3;$B$1;$A8) with B3=0,999 and B1=1e-9 and A8=1,5. The result > should be shown as 1,00000000000000E+000 but it is shown as > 10,00000000000000E+000. I couldn't reproduce that, using a number format code 0.00000000000000E+000 Did you do something different? > It is really a display problem and not a problem of my function, because > the result of the formula =BETADIST(B$3;$B$1;$A8)+1 is correctly shown > as 2,00000000000000E+000. > > To see the problem, you need my interpr3.cxx and the interpr6.cxx from > cws DEV300/odff04. You cannot use the normal version, because there the > function is not accurate enough to get that problem. I tried in odff04, no problem with Linux .pro and non-pro and Windows .pro builds. > I have simulate the algorithm with MuPad, a high precision CAS. There I > get the value 0.9999999999999993332933300668465 (first digits) for my > result. But I don't know which value OOo has internally. For the raw unformatted result you could set a breakpoint where BETADIST pushes its value and display it in the debugger or analyze the IEEE 754 double bits. > Does anyone know the reason or like to have a look at it or show me a > way to get the root of the problem? If it turns out to be really a display problem, it's probably the number formatter to look at, svtools/source/numbers/zformat.cxx method SvNumberformat::GetOutputString(double fNumber,...) the case NUMBERFORMAT_SCIENTIFIC However, as I can't reproduce the problem here I'd consider that the very last resort. Maybe rebuilding the zformat.obj object file, linking and replacing svtmi.dll in the installation already helps? Eike -- OOo/SO Calc core developer. Number formatter stricken i18n transpositionizer. SunSign 0x87F8D412 : 2F58 5236 DB02 F335 8304 7D6C 65C9 F9B5 87F8 D412 OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS Please don't send personal mail to the [EMAIL PROTECTED] account, which I use for mailing lists only and don't read from outside Sun. Use [EMAIL PROTECTED] Thanks.
pgpQPnpaWFjQA.pgp
Description: PGP signature
