kylebarron commented on code in PR #6157:
URL: https://github.com/apache/arrow-rs/pull/6157#discussion_r1729247546
##########
parquet/src/arrow/async_reader/metadata.rs:
##########
@@ -52,7 +51,44 @@ impl<F: MetadataFetch> MetadataLoader<F> {
/// Create a new [`MetadataLoader`] by reading the footer information
///
/// See [`fetch_parquet_metadata`] for the meaning of the individual
parameters
- pub async fn load(mut fetch: F, file_size: usize, prefetch: Option<usize>)
-> Result<Self> {
+ pub async fn load(mut fetch: F, prefetch: Option<usize>) -> Result<Self> {
+ let suffix =
fetch.fetch(GetRange::Suffix(prefetch.unwrap_or(8))).await?;
Review Comment:
I don't have a preference myself whether the default `load` uses a suffix
request or not. As @H-Plus-Time noted above, we're looking for consensus on
this.
> Aspects that need consensus:
>
> - which of the two options (suffix, or non-suffix) gets the load method
name (i.e. which is the default), and what the non-default method name should
be.
Separately,
> in most cases, we already have the file size from ListObjects or other
metadata services
this seems to depend heavily on your use case. In my case I rarely have this
information already.
> include it in the next minor version, avoiding a breaking change
This is moot anyways, because the next release is breaking, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
