Rich-T-kid commented on code in PR #10128:
URL: https://github.com/apache/arrow-rs/pull/10128#discussion_r3425201511
##########
arrow-ipc/src/writer.rs:
##########
@@ -462,16 +470,18 @@ impl IpcDataGenerator {
Ok(())
}
+ // Collect all dicts that need a dictionary message i.e. ones that were
+ // either not in the tracker previously or ones that were but need a
+ // replacement or delta.
#[allow(clippy::too_many_arguments)]
Review Comment:
Is it possible to remove the lint bypass? I know its not tied directly to
this PR but it'd be nice to incrementally clean up this file.
##########
arrow-ipc/src/writer.rs:
##########
@@ -550,52 +554,54 @@ impl IpcDataGenerator {
write_options: &IpcWriteOptions,
compression_context: &mut CompressionContext,
) -> Result<(Vec<EncodedData>, EncodedData), ArrowError> {
- let encoded_dictionaries = self.encode_all_dicts(
- batch,
- dictionary_tracker,
- write_options,
- compression_context,
- )?;
+ let dictionaries = self.collect_all_dicts(batch, dictionary_tracker,
write_options)?;
+ let mut encoded_dictionaries = Vec::with_capacity(dictionaries.len());
+ for dict in dictionaries {
+ encoded_dictionaries.push(self.dictionary_batch_to_bytes(
+ dict,
+ write_options,
+ compression_context,
+ )?);
+ }
+ let mut fbb = FlatBufferBuilder::new();
Review Comment:
Nice, I did something similar in
[arrow-flight](https://github.com/apache/arrow-rs/pull/10137/changes#diff-e11595282032996fd43b9f4749bf13e67094fe5d76c9d7795a5233c763e2aadeR779)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
