Interesting. This works as expected on our system. Unidata 6.something on AIX. I wonder what is different in our settings.
:FLOAT.PRECISION FLOAT.PRECISION mode 3 :RUN FIXBP JH.TST BAL [0] BAL <= 0 Matches integer MCN BAL <= 0 John M. Horn | Solutions Engineer, Advisor Sr. | HealthLink | 314-989-6050 | jh...@healthlink.com -----Original Message----- From: owner-u2-us...@listserver.u2ug.org [mailto:owner-u2-us...@listserver.u2ug.org] On Behalf Of Wally Terhune Sent: Friday, May 01, 2009 8:00 AM To: u2-users@listserver.u2ug.org Subject: Re: [U2] Unidata calculations incorrect look up the ECL statement FLOAT.PRECISION Folks are often happiest with FLOAT.PRECISION 4 Wally Terhune U2 Support Architect IBM Information Management Software Tel: (303) 773-7969 T/L 656-7969 Mobile: (303) 807-6222 Email: wal...@us.ibm.com http://www.ibm.com/software/data/u2/support From: "Martin Hutchinson" <m.hutchin...@webcontrolltd.co.uk> To: "u2 user group" <u2-users@listserver.u2ug.org> Date: 05/01/2009 05:21 AM Subject: [U2] Unidata calculations incorrect Sent by: owner-u2-us...@listserver.u2ug.org Hi All, We have Unidata 7.2 running on Windows 2008 server making incorrect calculations when numbers are close to zero. I understand there is a 'WIDEZERO' parameter on Univers but I don't think there is an equivalent on Unidata. Below we have an example program and output. 001: BAL = 856.19 002: VAL1 = 287.31 003: VAL2 = 568.88 004: BAL = BAL - VAL1 005: BAL = BAL - VAL2 006: * 007: CRT "BAL [":BAL:"]" 008: * 009: IF BAL > 0 THEN 010: CRT "BAL > 0" 011: END ELSE 012: CRT "BAL <= 0" 013: END 014: * 015: IF INT(BAL) = BAL THEN 016: CRT "Matches integer" 017: END ELSE 018: CRT "Does not match integer" 019: END 020: * 021: IF ICONV(BAL,"MCN") > 0 THEN 022: CRT "MCN BAL > 0" 023: END ELSE 024: CRT "MCN BAL <= 0" 025: END When run this gives: BAL [0] BAL > 0 Does not match integer MCN BAL <= 0 Has anyone come across this one or have any ideas? regards Martin Hutchinson ________________________ Web Control (UK) Ltd Atlantic House, Michigan Drive, Tongwell, Milton Keynes, MK15 8HQ T: +44(0)1908 216700 x4 F: +44(0)1908 216300 http://www.webcontrolltd.co.uk Web Control (UK) Ltd is a company registered in England and Wales with company number 4118558. VAT number: 770 3363 38 ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ [demime 1.01d removed an attachment of type image/gif which had a name of graycol.gif] [demime 1.01d removed an attachment of type image/gif which had a name of ecblank.gif] ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information or otherwise protected by law. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
