In the specification text of NV_vertex_program1_1, the upper limit of the RCC instruction is written as 1.884467e+19 in scientific notation, but as 0x5F800000 in binary. But the binary version translates to 1.84467e+19 rather than 1.884467e+19 in scientific notation.
Since the lower-limit equals 2^-64 and the binary version equals 2^+64, let's assume the value in scientific notation is a typo and implement this using the value from the binary version instead. Signed-off-by: Erik Faye-Lund <kusmab...@gmail.com> --- src/gallium/auxiliary/tgsi/tgsi_exec.c | 4 ++-- src/gallium/docs/source/tgsi.rst | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/gallium/auxiliary/tgsi/tgsi_exec.c b/src/gallium/auxiliary/tgsi/tgsi_exec.c index 96809cd..55da60a 100644 --- a/src/gallium/auxiliary/tgsi/tgsi_exec.c +++ b/src/gallium/auxiliary/tgsi/tgsi_exec.c @@ -915,9 +915,9 @@ micro_rcc(union tgsi_exec_channel *dst, float recip = 1.0f / src->f[i]; if (recip > 0.0f) - dst->f[i] = CLAMP(recip, 5.42101e-020f, 1.884467e+019f); + dst->f[i] = CLAMP(recip, 5.42101e-020f, 1.84467e+019f); else - dst->f[i] = CLAMP(recip, -1.884467e+019f, -5.42101e-020f); + dst->f[i] = CLAMP(recip, -1.84467e+019f, -5.42101e-020f); } } diff --git a/src/gallium/docs/source/tgsi.rst b/src/gallium/docs/source/tgsi.rst index 0501aca..be42572 100644 --- a/src/gallium/docs/source/tgsi.rst +++ b/src/gallium/docs/source/tgsi.rst @@ -424,7 +424,7 @@ XXX cleanup on aisle three .. math:: - dst = (1 / src.x) > 0 ? clamp(1 / src.x, 5.42101e-020, 1.884467e+019) : clamp(1 / src.x, -1.884467e+019, -5.42101e-020) + dst = (1 / src.x) > 0 ? clamp(1 / src.x, 5.42101e-020, 1.84467e+019) : clamp(1 / src.x, -1.84467e+019, -5.42101e-020) .. opcode:: DPH - Homogeneous Dot Product -- 1.8.5.3 _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/mesa-dev