On Fri, 22 Sep 2023 21:32:26 GMT, Joe Darcy <da...@openjdk.org> wrote:
> For FDLIBM tan, the stated error in the Math.tan spec is 1 ulp and the FDLIBM > sources say tan is "nearly rounded," which could reasonably be interpreted as > meaning within 1 ulp. However, the reported error by the paper is 1.02 ulps. > > As you note here, the current testing methodology can only really deal with > at most a 1 ulp error, assuming the expected value is at the lower endpoint > of the range. > > To avoid any lurking errors in the FDLIBM port to Java, I generated the > expected numbers running jshell on JDK 20, which uses the mostly C-based > FDLIBM. For a number of cases I had to decrement the expected value for the > test to pass against Math and StrictMath. The decremented value and its > successor may or may not bracket the exact value; I didn't verify that. > > In other words, there may be other bugs in one or both math libraries the > test is detecting. > > So far, I've only tried running the test locally, but this would need a > cross-platform run being before pushed to cover all the intrinsics that may > be in use on the full set of supported platforms. Subsequently, the allowable error bound for tan was bumped up to 1.25 ulps to cover the observed 1.02 ulp error under JDK-8326530: Widen allowable error bound of Math.tan. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/15879#discussion_r1501122082
