To compare to zero:
EPSILON = .0001 // how close do we need to be
if (abs(aFloatingPointValue) < EPSILON)...
Ken
On 12/31/2010 5:40 AM, Michael Büsch wrote:
> On Fri, 2010-12-31 at 10:11 +0000, Chris Morley wrote:
>> So I found one problem with the g10 l2 patch :
>> p-1 is accepted as p0 even if I check for p< 0
>> The reason is when the p number is converted to an integer:
>>
>> p_int = (int) (block->p_number + 0.0001);
>>
>> -1 +.0001 is (rightfully) truncated to zero.
>> If I remove the +0.0001 then -1 stays -1 and the error is
>> caught.
>> My question is the reason for the 0.0001 and can I remove
>> it without problems ?
>>
> Well, no. The reason for it seems to be that floating point
> numbers come with some inaccuracies.
> So the number 1 in floating point might be 1.0000001 or 0.99999998
> or something like that, depending on the CPU architecture
> and/or the operations that were done to the number before.
> However, when converted to int the 1.0000001 converts as 1 and
> the 0.99999998 converts as 0. And thus we have to add a "safe" value
> to it before casting it. 0.0001 is pretty safe for doubles and
> also seems to be safe for floats.
> This is also the reason why floating point numbers cannot be
> compared with ==. They often wouldn't match, even if it's
> actually the same number, due to inaccuracies.
>
--
Kenneth Lerman
55 Main Street
Newtown, CT 06470
203-426-3769
------------------------------------------------------------------------------
Learn how Oracle Real Application Clusters (RAC) One Node allows customers
to consolidate database storage, standardize their database environment, and,
should the need arise, upgrade to a full multi-node Oracle RAC database
without downtime or disruption
http://p.sf.net/sfu/oracle-sfdevnl
_______________________________________________
Emc-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/emc-developers
