Xuanwo commented on code in PR #6157:
URL: https://github.com/apache/arrow-rs/pull/6157#discussion_r1724380135
##########
parquet/src/arrow/async_reader/metadata.rs:
##########
@@ -52,7 +51,44 @@ impl<F: MetadataFetch> MetadataLoader<F> {
/// Create a new [`MetadataLoader`] by reading the footer information
///
/// See [`fetch_parquet_metadata`] for the meaning of the individual
parameters
- pub async fn load(mut fetch: F, file_size: usize, prefetch: Option<usize>)
-> Result<Self> {
+ pub async fn load(mut fetch: F, prefetch: Option<usize>) -> Result<Self> {
+ let suffix =
fetch.fetch(GetRange::Suffix(prefetch.unwrap_or(8))).await?;
Review Comment:
Hi, thanks a lot for this effort. However, using suffix fetch by default may
break users on storage services that don't support this, like azblob. Have you
considered not changing `load`, but adding a new API called
`load_without_size`, `load_with_suffix_fetch`, or something similar?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
