This follows up on the review of the SFF-8472 Rx power calibration fix,
where the automated review pointed out the remaining issues in
sff_8472_calibration().
Calibration formula 1 (temperature, voltage, Tx bias, Tx power) scales
the 16-bit fields in place by a slope of up to 255.996 and adds a signed
16-bit offset. A result out of the field range is undefined behavior
for the float to integer conversion; a negative sum wraps around.
UBSan reports float-cast-overflow on randomly generated externally
calibrated SFP dumps.
Patch 1 computes formula 1 in double precision and saturates the result
to the field range, as already done for the Rx power. It is intended
for stable.
Patch 2 rounds the calibrated values to the nearest integer instead
of truncating them: RX_PWR(1) = 0.7 is stored as 0.69999999 and
a raw reading of 1000 gave 699 instead of 700.
Patch 3 adds unit tests for saturation at both ends and for rounding.
Based on dpdk-next-net for-main.
Depends-on: series-39455 ("ethdev: fix SFF-8472 external Rx power calibration")
Roman Khromenok (3):
ethdev: fix SFF-8472 calibration overflow
ethdev: round SFF-8472 calibrated values
test: check SFF-8472 calibration saturation and rounding
app/test/test_ethdev_module_eeprom.c | 96 ++++++++++++++++++++++++++++
lib/ethdev/sff_8472.c | 55 +++++++++++-----
2 files changed, 134 insertions(+), 17 deletions(-)
--
2.47.3
