Re: Precision Error?

[Sergey via Digitalmars-d-learn]

On Sunday, 6 October 2024 at 21:09:22 UTC, WhatMeWorry` wrote:

    float computeToFitInWindow(uint rows, uint cols)
    {
        // code removed ...
        writeln(hexWidth);
        return hexWidth;
    }


    unittest
    {

assert(computeToFitInWindow(1, 1) == 2.0f, "This assert failed"); assert(computeToFitInWindow(3, 3) == 0.8f, "This assert failed"); assert(computeToFitInWindow(2, 2) == 1.14286f, "This assert failed"); // Line 74

    }


2
0.8
1.14286
main.d(74): [unittest] This assert failed

Can someone tell me why line 74 fails? Aren't they both 1.14286? I presume it has to do with precision. Is there a technique to address this issue.

https://dlang.org/library/std/math/operations/is_close.html