Shawn If you are still getting issues and the issue is down to the division itself, you might want to try switching to string maths (HELP BASIC SDIV). It is slower but may give more accurate results in some cases.
Remember that fractional numbers are not represented correctly in binary, and thus any floating point maths (whatever platform) are subject to rounding errors. Most financial systems for example scale up/down to integral values when performing calculations to avoid these issues. The fact that one of the numbers is showing as 0.3999999 suggests that it probably has more digits to follow... Brian -----Original Message----- From: u2-users-boun...@listserver.u2ug.org [mailto:u2-users-boun...@listserver.u2ug.org] On Behalf Of Shawn Hayes Sent: 18 June 2010 9:26 PM To: u2-users@listserver.u2ug.org Subject: [U2] Division Issues Hello, I ran into an issue when dividing 2 numbers. I debugged through the first program and here is the result... TEST1: 27: IF LOAD.VALUE THEN :: S TEST1: 28: ANS = TOP.VALUE/BOTTOM.VALUE ::S ::TOP.VALUE/ NUMBER: 0.399999999 :: BOTTOM.VALUE/ NUMBER: 0.006554707 :: ANS/ NUMBER: 61.02484472 I created a little test program to divide the same 2 numbers and I came up with this... TEST.DIV: 3: NUM1 = 0.399999999 :: S TEST.DIV: 4: NUM2 = 0.006554707 :: S TEST.DIV: 5: PRINT NUM1/NUM2 :: S 61.024848098 As you can see 61.024848098 does not equal 61.02484472? I have messed around with the PRESICION statement and I still have not gotten this to work? Can someone advise? Thanks!!! 'We act as though comfort and luxury were the chief requirements of life, when all that we need to make us happy is something to be enthusiastic about.' _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users Internal Virus Database is out of date. Checked by AVG - www.avg.com Version: 9.0.819 / Virus Database: 271.1.1/2911 - Release Date: 06/01/10 19:25:00 _______________________________________________ U2-Users mailing list U2-Users@listserver.u2ug.org http://listserver.u2ug.org/mailman/listinfo/u2-users
