Manu Zhang created AVRO-4360:
--------------------------------
Summary: [C++] Float and double binary codecs use native byte
order on big-endian hosts
Key: AVRO-4360
URL: https://issues.apache.org/jira/browse/AVRO-4360
Project: Apache Avro
Issue Type: Bug
Components: c++
Reporter: Manu Zhang
Avro C++ binary float/double encoding and decoding copy native memory bytes
directly, while the Avro binary format requires little-endian IEEE 754 bytes.
On a big-endian host such as s390x:
* Encoding float 1.0 produces 3f 80 00 00 instead of 00 00 80 3f.
* Encoding double 1.0 produces 3f f0 00 00 00 00 00 00 instead of 00 00 00 00
00 00 f0 3f.
* Decoding valid little-endian Avro bytes produces incorrect values.
Affected methods:
* lang/c++/impl/BinaryEncoder.cc: BinaryEncoder::encodeFloat and encodeDouble.
* lang/c++/impl/BinaryDecoder.cc: BinaryDecoder::decodeFloat and decodeDouble.
The defect is present on current main, commit
28cb08c15543c8c2dd8d186db2029bfe760940a5. An encode/decode round trip on a
single big-endian host can conceal it because the two incorrect operations
cancel each other out.
Reproduction and validation:
* Independent fixed wire-byte tests for float and double, covering positive and
negative zero, 1.0, -2.5, subnormal and normal limits, largest finite values,
infinities, and quiet NaN payloads.
* Stream chunk sizes of 1, 3 and 16 bytes.
* GCC 14 s390x cross-build under QEMU: the original codec fails 120 of 132
assertions; the corrected codec passes all 132. This is emulation rather than
physical s390x hardware.
* Complete CodecTests passes on native macOS and Linux ARM64: 852 test cases
and 103,659 assertions on each.
Expected behavior: encode and decode the required little-endian wire bytes
while preserving raw IEEE 754 bits.
Proposed fix uses C++20 bit_cast and endian-aware byte reversal in all four
methods. Regression tests check each direction independently.
Pull request: https://github.com/apache/avro/pull/4043
Specification: https://avro.apache.org/docs/1.12.0/specification/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
