In the specification text of NV_vertex_program1_1, the upper limit of the RCC instruction is written as 1.884467e+19 in scientific notation, but as 0x5F800000 in binary. But the binary version translates to 1.84467e+19 rather than 1.884467e+19 in scientific notation.
Since the lower-limit equals 2^-64 and the binary version equals 2^+64, let's assume the value in scientific notation is a typo and implement this using the value from the binary version instead. Signed-off-by: Erik Faye-Lund <kusmab...@gmail.com> --- I've also e-mailed NVIDIA's contact as listed in the specification, but I haven't gotten any reply yet. I think it's pretty obvious, though. I seriously doubt anyone implemented the old limit, as it would require quite a bit more gates AFAIK. src/gallium/auxiliary/tgsi/tgsi_exec.c | 8 ++++---- src/gallium/docs/source/tgsi.rst | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/src/gallium/auxiliary/tgsi/tgsi_exec.c b/src/gallium/auxiliary/tgsi/tgsi_exec.c index 3d37eaa..7d35aeb 100644 --- a/src/gallium/auxiliary/tgsi/tgsi_exec.c +++ b/src/gallium/auxiliary/tgsi/tgsi_exec.c @@ -915,8 +915,8 @@ micro_rcc(union tgsi_exec_channel *dst, float recip = 1.0f / src->f[i]; if (recip > 0.0f) { - if (recip > 1.884467e+019f) { - dst->f[i] = 1.884467e+019f; + if (recip > 1.84467e+019f) { + dst->f[i] = 1.84467e+019f; } else if (recip < 5.42101e-020f) { dst->f[i] = 5.42101e-020f; @@ -926,8 +926,8 @@ micro_rcc(union tgsi_exec_channel *dst, } } else { - if (recip < -1.884467e+019f) { - dst->f[i] = -1.884467e+019f; + if (recip < -1.84467e+019f) { + dst->f[i] = -1.84467e+019f; } else if (recip > -5.42101e-020f) { dst->f[i] = -5.42101e-020f; diff --git a/src/gallium/docs/source/tgsi.rst b/src/gallium/docs/source/tgsi.rst index 0501aca..be42572 100644 --- a/src/gallium/docs/source/tgsi.rst +++ b/src/gallium/docs/source/tgsi.rst @@ -424,7 +424,7 @@ XXX cleanup on aisle three .. math:: - dst = (1 / src.x) > 0 ? clamp(1 / src.x, 5.42101e-020, 1.884467e+019) : clamp(1 / src.x, -1.884467e+019, -5.42101e-020) + dst = (1 / src.x) > 0 ? clamp(1 / src.x, 5.42101e-020, 1.84467e+019) : clamp(1 / src.x, -1.84467e+019, -5.42101e-020) .. opcode:: DPH - Homogeneous Dot Product -- 1.8.5.3 _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/mesa-dev