wgtmac commented on PR #48345: URL: https://github.com/apache/arrow/pull/48345#issuecomment-5042439463
The serialized types and accepted vector-size range appear to diverge from the proposed parquet-format specification. The spec defines `num_exceptions` and exception positions as `uint16`, but this implementation stores both as signed `int16_t`. A valid vector with `log_vector_size = 15` can contain 32,768 values; if all values are exceptions, `num_exceptions = 32768` overflows `int16_t` and may lead to negative size calculations during decoding. Also, the decoder currently accepts `log_vector_size` values 1 and 2, while the proposed spec restricts the field to `[3, 15]`. Could we align the wire types and validation with the spec, and add boundary interoperability tests for 32,768 exceptions and the minimum accepted vector size before claiming the wire formats match? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
