EmilyMatt commented on code in PR #8930:
URL: https://github.com/apache/arrow-rs/pull/8930#discussion_r2683751849
##########
arrow-avro/src/reader/async_reader/async_file_reader.rs:
##########
@@ -0,0 +1,18 @@
+use crate::reader::async_reader::DataFetchFutureBoxed;
+use std::ops::Range;
+
+/// A broad generic trait definition allowing fetching bytes from any source
asynchronously.
+/// This trait has very few limitations, mostly in regard to ownership and
lifetime,
+/// but it must return a boxed Future containing [`bytes::Bytes`] or an error.
+pub trait AsyncFileReader: Send + Unpin {
+ /// Fetch a range of bytes asynchronously using a custom reading method
+ fn fetch_range(&mut self, range: Range<u64>) -> DataFetchFutureBoxed;
+
+ /// Fetch a range that is beyond the originally provided file range,
+ /// such as reading the header before reading the file,
+ /// or fetching the remainder of the block in case the range ended before
the block's end.
+ /// By default, this will simply point to the fetch_range function.
+ fn fetch_extra_range(&mut self, range: Range<u64>) -> DataFetchFutureBoxed
{
+ self.fetch_range(range)
+ }
+}
Review Comment:
1. Indeed, but I'd like something like fetching the metadata/header in this
case to be separate, parquet does this as well.
2. I agree with that, I will consider how to best approach this^^
3. while it's true that the trait does not need this restriction, it is
necessary in order to write the actual code, the parquet reader also has
<T: AsyncFileReader + Send + 'static>
Otherwise you simply could not use the underlying async readers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
