Ted-Jiang commented on code in PR #2116:
URL: https://github.com/apache/arrow-rs/pull/2116#discussion_r926198825
##########
parquet/src/arrow/record_reader/mod.rs:
##########
@@ -120,9 +96,25 @@ where
/// Set the current page reader.
pub fn set_page_reader(&mut self, page_reader: Box<dyn PageReader>) ->
Result<()> {
- self.column_reader = Some(GenericColumnReader::new(
+ let descr = &self.column_desc;
+ let values_decoder = CV::new(descr);
+
+ let def_level_decoder = (descr.max_def_level() != 0).then(|| {
+ DefinitionLevelBufferDecoder::new(
+ descr.max_def_level(),
+ packed_null_mask(descr),
Review Comment:
I think choose values_decoder type only according to `column_desc` it's file
level not page level.
So i think we don't need check every time in `set_page_reader` am i right? 🤔
##########
parquet/src/arrow/record_reader/mod.rs:
##########
@@ -120,9 +96,25 @@ where
/// Set the current page reader.
pub fn set_page_reader(&mut self, page_reader: Box<dyn PageReader>) ->
Result<()> {
- self.column_reader = Some(GenericColumnReader::new(
+ let descr = &self.column_desc;
+ let values_decoder = CV::new(descr);
+
+ let def_level_decoder = (descr.max_def_level() != 0).then(|| {
+ DefinitionLevelBufferDecoder::new(
+ descr.max_def_level(),
+ packed_null_mask(descr),
Review Comment:
One column chunk has the same def_level_decoder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
