rok commented on issue #14923: URL: https://github.com/apache/arrow/issues/14923#issuecomment-1366851317
@mapleFU I am worried about this part of [the spec](https://github.com/apache/parquet-format/blob/master/Encodings.md#delta-encoding-delta_binary_packed--5): > If, in the last block, less than <number of miniblocks in a block> miniblocks are needed to store the values, the bytes storing the bit widths of the unneeded miniblocks are still present, their value should be zero, but readers must accept arbitrary values as well. I'm not sure how other encoder authors would interpret this, but I expected that uninitialized `bit_width_data[i]` would produce a random `int` that wouldn't cause problems roundtripping. My interpretation is probably wrong. Maybe we should propose to make the spec clearer or stricter? > So, once it reaches the last block, it will just read the last mini-block with data, the remaining buffer will not be read or checked. That sounds like a good idea! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
