MassivePizza commented on code in PR #10345:
URL: https://github.com/apache/arrow-rs/pull/10345#discussion_r3614449690
##########
parquet/src/compression.rs:
##########
@@ -534,22 +535,31 @@ mod zstd_codec {
output_buf: &mut Vec<u8>,
uncompress_size: Option<usize>,
) -> Result<usize> {
- let capacity = uncompress_size.unwrap_or_else(|| {
- // Get the decompressed size from the zstd frame header
- zstd::zstd_safe::get_frame_content_size(input_buf)
- .ok()
- .flatten()
- .unwrap_or(input_buf.len() as u64 * 4) as usize
+ let offset = output_buf.len();
+ let len = uncompress_size.unwrap_or_else(|| {
Review Comment:
I thought about this and we may want to simply replace `upper_bound` with
`get_frame_content_size`;
* We restore the behavior of decoding at most one frame or
`uncompress_size`, whichever is smaller.
* `upper_bound` only ever returns more than `get_frame_content_size`, making
it redundant after inlining `decompress(...) -> Vec`.
* `get_frame_content_size` is not gated behind `experimental` feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
