alamb commented on code in PR #9447:
URL: https://github.com/apache/arrow-rs/pull/9447#discussion_r2966117892
##########
parquet/src/encodings/levels.rs:
##########
@@ -26,6 +26,7 @@ use crate::util::bit_util::{BitWriter, ceil,
num_required_bits};
/// Computes max buffer size for level encoder/decoder based on encoding, max
/// repetition/definition level and number of total buffered values (includes
null
/// values).
+#[allow(dead_code)]
Review Comment:
How about just removing it?
##########
parquet/src/column/writer/mod.rs:
##########
@@ -1051,23 +1060,19 @@ impl<'a, E: ColumnValueEncoder> GenericColumnWriter<'a,
E> {
let mut buffer = vec![];
if max_rep_level > 0 {
- buffer.extend_from_slice(
- &self.encode_levels_v1(
- Encoding::RLE,
- &self.rep_levels_sink[..],
- max_rep_level,
- )[..],
+ let encoder = mem::replace(
+ &mut self.rep_levels_encoder,
Review Comment:
This path recreates the encoders each time (and thus probably allocates, etc)
It seems like the old code path does the same thing (in encode_levels_v1, ..)
@HippoBaro would you be open to exploring adding a "clear" method to the
encoder now that you have it encapsulated to save the allocation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
