rustyconover opened a new issue, #970:
URL: https://github.com/apache/arrow-nanoarrow/issues/970
The IPC encoder truncates metadata keys and values at the first NUL byte.
`ArrowIpcEncodeMetadata()` (`src/nanoarrow/ipc/encoder.c`) writes each
`KeyValue` with flatcc's `KeyValue_key_create_strn()` /
`KeyValue_value_create_strn()`. The `_create_strn()` variants are
`flatcc_builder_create_string(B, s, strnlen(s, max_len))`, so they stop at the
first NUL even though the length is known. NUL (U+0000) is valid UTF-8, and
both nanoarrow's packed metadata and flatbuffers strings carry an explicit
length. As a result, a key or value with an embedded NUL is silently cut short
in Schema, Field and Message `custom_metadata`.
For example, a field metadata pair `("field\0key", "\0field\0value")`
round-trips through the encoder and decoder as `("field", "")`. The decoder
already reads keys and values by length, so only the encoder is affected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
