jhorstmann commented on code in PR #11136:
URL: https://github.com/apache/arrow-rs/pull/11136#discussion_r4149696642
##########
parquet/src/encodings/decoding.rs:
##########
@@ -2190,6 +2190,35 @@ mod tests {
test_byte_stream_split_decode::<Int64Type>(data, -1);
}
+ #[test]
+ fn test_byte_stream_split_uses_encoded_value_count() {
+ let mut decoder = ByteStreamSplitDecoder::<DoubleType>::new();
+
+ decoder.set_data(Bytes::from(vec![0; 64]), 63).unwrap();
Review Comment:
> BYTE_STREAM_SPLIT stores only non-null values
That is true of all parquet encodings, so I would assume it is handled
correctly by other encoders. Maybe the length checks could be done better in
`get` instead of `set_data`, at that point the caller provides a buffer that
should be sized correctly for the non-null values.
A similar check is probably also needed in
`VariableWidthByteStreamSplitDecoder`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
