pedrobtz opened a new pull request, #989: URL: https://github.com/apache/arrow-nanoarrow/pull/989
`ArrowIpcDictionariesInit()` supports several fields sharing one dictionary id. It makes one `ArrowIpcDictionary` per unique id and decodes its batches with the value type of the first field that has that id. Nothing checks that the other fields with the id have the same value type. If they don't, `ArrowIpcDecoderWalkSetArrayView()` views the decoded dictionary with the second field's type. For a `string` dictionary read by a field whose values are `large_string`, the 4-byte offsets are read as 8-byte ones. ASan reports a heap-buffer-overflow in `ArrowArrayViewValidateDefault()`. `ArrowIpcArrayStreamReader` hits this on a schema message with two such fields, followed by a dictionary batch and a record batch. `ArrowIpcDictionariesInit()` now returns `EINVAL` when two fields share an id but not a value type. The comparison covers formats, children and nested dictionaries, not names, flags or metadata. Shared dictionaries with the same value type work as before. The new `NanoarrowIpcSharedDictionaryValueTypes` test initialises dictionaries for two fields with id 0, first both `string`, then `string` and `large_string`. It expects success and then `EINVAL`. Without the fix the second case succeeds. I also read the stream that crashed through `ArrowIpcArrayStreamReader`. It now fails with the new message instead of reading out of bounds. Found by fuzzing the IPC reader in the [zuarrow](https://github.com/pedrobtz/zuarrow) R package, which vendors nanoarrow. `ctest` passes in a plain Debug build and with ASan and UBSan (410 tests each). The changed files are formatted with the pinned clang-format 16.0.6. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
