Adrian, I was able to reproduce your error. I believe your assumption is correct. The internal value probably is greater than 1, which would be an invalid argument as cosine is between -1 and 1. I put the following lines in before adding VAR1 and VAR3 together and the error was resolved. VAR1 = (VAR1 * 100000000000000) / 100000000000000 VAR3 = (VAR3 * 100000000000000) / 100000000000000 (round VAR1 and VAR3 to 14 digits before adding) I don't have any other ideas how you would get around it, unless you could carry the precision out far enough to determine the value would be outside of the range. Of course, in UV you are already at the highest precision with it set to 14, so you might have to go outside of UV to do this. Mike
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Womack, Adrian Sent: Thursday, April 14, 2005 8:56 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] [UV] ACOS problem (floating point numbers) Mike, A CRT of ACOS(1) and ACOS(TEMP) where TEMP = 1 also returns zero for me. That's not the problem. Try the code in my example, the variable VAR4 displays as 1 - but is probably slightly higher than 1 internally. It's in this situation where I get the error message. Adrian -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mike Dallaire Sent: Thursday, 14 April 2005 9:54 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] [UV] ACOS problem (floating point numbers) Adrian, I didn't really misunderstand what you were trying to do, I was just clarifying a usual problem when dealing with trig functions. However, I did misunderstand how UV is using ACOS. ACOS or Arc Cosine is the mathematical inverse of Cosine. In UV though, it is treated like cos -1 (the inverse function on most calculator's cos key). This value is not the mathematical inverse. It returns the numeric value for what the number you pass would be the COS of. But enough math mumbo, jumbo. I ran a simple test on our UV 10.1.3 on Win 2000 box. I just did a crt of ACOS(1) and ACOS(TEMP) where TEMP = 1 and both displayed 0. I did not receive an error at all. I tried it at true TCL and in SB+. Are you sure you are not missing something? What are you running on? Mike -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Womack, Adrian Sent: Wednesday, April 13, 2005 8:51 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] [UV] ACOS problem (floating point numbers) Mike, I think you misunderstood the problem. We're not trying to use ACOS() on zero, but on a calculated value that happens to be 1. CRT ACOS(1) returns zero but CRT ACOS(VAR4) produces the error message, even though VAR4 prints as "1", has a LEN() of 1, is equal to 1 (according to IF VAR1 = 1 THEN CRT "TRUE"). AdrianW -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Mike Dallaire Sent: Wednesday, 13 April 2005 8:48 PM To: u2-users@listserver.u2ug.org Subject: RE: [U2] [UV] ACOS problem (floating point numbers) Adrian, ACOS is the inverse value of COS. At any point when COS equates to 0, ACOS will be undefined, as it would be 1/0. When calculating distances (assuming east-west being your x axis and north-south being your y axis), COS would always be 0 when there is no east or west vector involved (the distance traveled east or west is 0, or the sum of the vectors is 0, depending upon your calculation) Adding the null character may fool the processor into not seeing the value as 0, I am not sure. However, anytime you are using trigonometric functions you need to take into account values where they can be undefined. SIN and COS are the only functions defined for all values, IIRC. HTH, Mike -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Womack, Adrian Sent: Wednesday, April 13, 2005 4:55 AM To: u2-users@listserver.u2ug.org Subject: [U2] [UV] ACOS problem (floating point numbers) We have some code that is calculating distances between points, using SIN(), COS() and ACOS(). Occasionally we get the error: "Argument domain error in arc-cosine, zero returned" This seems to be occurring when two floating point numbers are added together and the result is exactly 1 (or at least seems to be exactly 1), I assume internally the number is slightly over 1. Here's some example code: PRECISION 14 FROM.LAT = -31.295833333 FROM.LONG = 119.651944444 DEST.LAT = -31.295833333 DEST.LONG = 119.651944444 VAR1 = SIN(FROM.LAT) * SIN(DEST.LAT) VAR2 = (DEST.LONG - FROM.LONG) VAR3 = COS(FROM.LAT) * COS(DEST.LAT) * COS(VAR2) VAR4 = VAR1 + VAR3 CRT "VAR1 = ":VAR1 CRT "VAR2 = ":VAR2 CRT "VAR3 = ":VAR3 CRT "VAR4 = ":VAR4 TEMP1 = ACOS(VAR4) TEMP2 = ACOS(VAR4:"") VAR4 is displayed as "1", but when TEMP1 is assigned we get the above error message. When TEMP2 is assigned we get no error (i.e. the temporary variable passed to the second ACOS function was a string not a number). Maybe concatenating an empty string during the ACOS() call is the fix for this but it's not really ideal. Anyone have any comments on this? And what happens on your machine? BTW our widezero config setting is 3dc00000 AdrianW DISCLAIMER: Disclaimer. This e-mail is private and confidential. If you are not the intended recipient, please advise us by return e-mail immediately, and delete the e-mail and any attachments without using or disclosing the contents in any way. The views expressed in this e-mail are those of the author, and do not represent those of this company unless this is clearly indicated. You should scan this e-mail and any attachments for viruses. This company accepts no liability for any direct or indirect damage or loss resulting from the use of any attachments to this e-mail. ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ DISCLAIMER: Disclaimer. This e-mail is private and confidential. If you are not the intended recipient, please advise us by return e-mail immediately, and delete the e-mail and any attachments without using or disclosing the contents in any way. The views expressed in this e-mail are those of the author, and do not represent those of this company unless this is clearly indicated. You should scan this e-mail and any attachments for viruses. This company accepts no liability for any direct or indirect damage or loss resulting from the use of any attachments to this e-mail. ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ DISCLAIMER: Disclaimer. This e-mail is private and confidential. If you are not the intended recipient, please advise us by return e-mail immediately, and delete the e-mail and any attachments without using or disclosing the contents in any way. The views expressed in this e-mail are those of the author, and do not represent those of this company unless this is clearly indicated. You should scan this e-mail and any attachments for viruses. This company accepts no liability for any direct or indirect damage or loss resulting from the use of any attachments to this e-mail. ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/ ------- u2-users mailing list u2-users@listserver.u2ug.org To unsubscribe please visit http://listserver.u2ug.org/
