Ted-Jiang commented on code in PR #1791:
URL: https://github.com/apache/arrow-rs/pull/1791#discussion_r889867688
##########
parquet/src/file/metadata.rs:
##########
@@ -223,6 +223,7 @@ pub struct RowGroupMetaData {
num_rows: i64,
total_byte_size: i64,
schema_descr: SchemaDescPtr,
+ // Todo add filter result -> row range
Review Comment:
Yes, you are right. page index stored in file-meta level.
My thought is read less pageIndex after rowgroup filter
https://github.com/apache/arrow-rs/blob/be388291036512b925796fdff50b39afdbceee20/parquet/src/file/serialized_reader.rs#L211-L224
https://github.com/apache/arrow-rs/blob/be388291036512b925796fdff50b39afdbceee20/parquet/src/file/serialized_reader.rs#L246-L249
So i want to read index here and insert it into RowGroupMetaData.
It was just a simple idea at first, maybe we can find a better way in the
process of implementation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
