alamb commented on PR #11157: URL: https://github.com/apache/arrow-rs/pull/11157#issuecomment-5919893002
> Another thought is we could divorce the page index parsing from the ParquetMetaData. We could add new functions to the decoders that return a PageIndex directly without installing it in the provider...leave that up to the user. Then the user could maintain control of the index and implement their own merge strategy. Kind of like how we envision a custom provider working. PageIndex just becomes a convenience provided to users who don't want to deal with maintaining their own parsing stack. We'd keep the replace logic currently implemented here for the existing API. Another thing to think about is how to (eventually) unify BloomFilter into this whole thing -- specifically if the BloomFilter should be part of ParquetMetadata or not. It is kind of nice to be able to pass around one ParquetMetadata thing that has pointers to the various metadata structures, but it is weird that it is not a 1:1 correspondence of wht is in the footer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
