rounding error
--------------
Key: CORE-6232
URL: http://tracker.firebirdsql.org/browse/CORE-6232
Project: Firebird Core
Issue Type: Bug
Components: API / Client Library
Affects Versions: 3.0.5
Environment: Win7
Reporter: at
Priority: Critical
In dialect 1of FB3 I've created two samples functions test_round1 and
test_round2. They give two DIFFERENT results 654.08 i 654.07. Why?
Arithmetically results should be the same 654.08.
I know that dialect1 have problems with precision and I should migrate to
dialect 3, but in then same dialect I should recived same results (even rounded
incorrectly).
set term ^ ;
create or alter function test_round1
returns double precision
as
declare variable x numeric(15,2);
declare variable y numeric(15,2);
begin
x= 68.85*9.50;
y = 1.0;
return round(x*y, 2);
end^
create or alter function test_round2
returns double precision
as
declare variable x numeric(15,2);
declare variable y numeric(15,2);
begin
x= 68.85;
y = 1.0*9.50;
return shudf_roundfloat(x*y, 2);
end^
set term ; ^
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
