Hi George,
As a general rule in programming, comparison of floating point values for
equality should be avoided. This is because, just as we cannot write the
number one third accurately in decimal notation, so the IEEE floating point
format used by computer systems cannot store numbers accurately. The example
that I use when teaching training courses is 14.2 which actually ends up as
something close to 14.19999999997.
UniVerse gets around this with a wonderful concept called "wide zero" that
says, when testing for equality of floating point numbers, they must be
within some specified value of being equal rather than strictly equal. The
default wide zero error tolerance, set in IEEE format with the WIDEZERO
configuration parameter, is 2.91 * 10^-11 (2^ 035) which is good for most
business applications but occasionally needs adjusting.
Unidata has a command, SET.WIDEZERO, to serve the same purpose but defaults
to 0.0 for backward comaptibility.
Martin Phillips
Ladybridge Systems Ltd
17b Coldstream Lane, Hardingstone, Northampton NN4 6DB, England
+44 (0)1604-709200
----- Original Message -----
From: "George Hammerle" <zhamme...@hubert.com>
To: <u2-users@listserver.u2ug.org>
Sent: Thursday, September 29, 2011 1:30 PM
Subject: [U2] The math just doesn't work.
Can anybody please help?
For some reason A + B does not equal C in the comparison below. Is there
any trick to get the comparisons to work properly?
Unidata 7.2 on Hp Unix 11+
Top of "TEST.COMP" in "RMH.MAIN", 13 lines, 263 characters.
*--: P
001: A = 3176.79
002: B = 106.19
003: C = 3282.98
004: D = 920.11
005: A = A + D
006: C = C + D
007: IF (A+B) # C THEN
008: CRT '(A+B) # C? YOU LIE'
009: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
010: END ELSE
011: CRT '(A+B) = C? YOU ROCK'
012: CRT 'A = ':A:', B = ':B:', (A+B) = ':(A+B):', C = ':C
013: END
Bottom.
*--: FIBR
Filed "TEST.COMP" in file "RMH.MAIN" unchanged.
Compiling Unibasic: /db1/ud1/PGM/RMH.MAIN/TEST.COMP in mode 'u'.
compilation finished
(A+B) # C? YOU LIE
A = 4096.9, B = 106.19, (A+B) = 4203.09, C = 4203.09
George Hammerle
Programming Dude
Hubert Company LLC.
9555 Dry Fork Road
Harrison, Ohio 45030
513-367-8974
zhammerle@hubertREMOVE_THIS.com
_______________________________________________
U2-Users mailing list
U2-Users@listserver.u2ug.org
http://listserver.u2ug.org/mailman/listinfo/u2-users
