I tried it again cutting and pasting it verify that I was not corrupting the
data. I also tried;
2156737.51 - 16123.85
Which netted the same result.
So from the other emails on this subject I can see that I definitely
misunderstood about DQSD's ability to perform calculations in a normal
manner. What a shame, and waiting for google to run the calc and return it
is a pill too. Is there no way to set DQSD to preform normal arithmetic
functions?
I'll admit I am lost on the way this is functioning, but to me the end
result is it works for some math and others it doesn't and therefore cant be
trusted. Perhaps on the functions list it should be made clear that this is
not doing standard arithmetic but binary. Or maybe I'm the only person out
here who looks at this and goes "What?!?!?!?!".
On 5/18/07, Kim Gräsman <[EMAIL PROTECTED]> wrote:
Hi Richard,
> Uhhh that doesnt sound right to me. I subtracted 3 different values
> from another value and got the same result twice.
Oops, I didn't see your results were identical for the first two cases.
I tried it here;
2156737.51 - 16123.81 => 2140613.6999999997
2156737.51 - 16123.83 => 2140613.6799999997 <-- Note: .6_7_999...
2156737.51 - 16123.82 => 2140613.69
Could you try it again to make sure you transcribed the values correctly?
> Also two of the results are flat out wrong.
Right, but they're pretty close to correct, which is what you get with
floating-point, if I understand things correctly.
> I could understand if I had used some kind of hokey notation but its
> a straight up math problem X-Y. When performed in a calculator
> application I dont get that result?
The calculator (at least the one in Windows) probably doesn't use
floating-point to store its numbers, but again, I'm a bit out of my
depth here.
Cheers,
- Kim
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
To unsubscribe visit:
https://lists.sourceforge.net/lists/listinfo/dqsd-users
DQSD-Users@lists.sourceforge.net
http://sourceforge.net/mailarchive/forum.php?forum_id=8601
--
Richard Ahlquist
Systems Analyst
Webmaster
http://www.patentlystupid.com
http://www.pcsites.com
http://www.unlimitedbs.com
-------------------------------------------------------------------------
This SF.net email is sponsored by DB2 Express
Download DB2 Express C - the FREE version of DB2 express and take
control of your XML. No limits. Just data. Click to get it now.
http://sourceforge.net/powerbar/db2/
_______________________________________________
To unsubscribe visit:
https://lists.sourceforge.net/lists/listinfo/dqsd-users
DQSD-Users@lists.sourceforge.net
http://sourceforge.net/mailarchive/forum.php?forum_id=8601
