CurtHagenlocher opened a new issue, #124:
URL: https://github.com/apache/parquet-testing/issues/124
The object `value_header` holds `field_offset_size_minus_one` in bits 0-1
and `field_id_size_minus_one` in bits 2-3. Writers compute the two
independently -- field ID width from the largest field ID in the object, offset
width from the size of the field data -- so the two routinely differ.
Every object example in `variant/` uses 1-byte field IDs and 1-byte field
offsets. When the two are equal, the two header fields hold the same value, so
a reader that reads them from each other's bit positions still decodes
correctly. The corpus therefore cannot detect that class of bug. arrow-dotnet
shipped exactly this bug and validated cleanly against this corpus the whole
time (apache/arrow-dotnet#420).
Two examples would close the gap:
- Few fields (1-byte field IDs) but more than 256 bytes of field data
(2-byte field offsets) -- header byte `0x06`.
- A metadata dictionary of more than 256 entries (2-byte field IDs)
containing a small object whose own field data is under 256 bytes (1-byte field
offsets) -- header byte `0x12`.
Related to #78 but not covered by it: #78 is framed around field *count* and
treats the two widths as one knob ("a 3 byte field id / offset"). The first
example above has only two fields. The second overlaps with #78's ">2^8
fields" case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
