xanderbailey commented on code in PR #2822:
URL: https://github.com/apache/iceberg-rust/pull/2822#discussion_r3669559123
##########
crates/iceberg/src/puffin/reader.rs:
##########
@@ -19,40 +19,59 @@ use tokio::sync::OnceCell;
use super::validate_puffin_compression;
use crate::Result;
-use crate::io::InputFile;
+use crate::encryption::EncryptedInputFile;
+use crate::io::{FileRead, InputFile};
use crate::puffin::blob::Blob;
use crate::puffin::metadata::{BlobMetadata, FileMetadata};
/// Puffin reader
pub struct PuffinReader {
- input_file: InputFile,
+ file_read: Box<dyn FileRead>,
+ file_length: u64,
file_metadata: OnceCell<FileMetadata>,
}
impl PuffinReader {
- /// Returns a new Puffin reader
- pub fn new(input_file: InputFile) -> Self {
+ /// Returns a new Puffin reader for an unencrypted file.
+ pub async fn new(input_file: InputFile) -> Result<Self> {
Review Comment:
Okay I gave this a shot and to make it lazy we'd need some fairly awkward
gymnastics. Unlike `ManifestWriterBuilder`, which consumes its `writer_future`
exactly once in `build`, `PuffinReader` reads through `&self` in both
`file_metadata()` and `blob()`, the future is consumed on poll, so a stored
`BoxFuture` can't back repeated reads without wrapping it in something like
`Mutex<Option<..>>` + a `OnceCell` to resolve-once-and-cache. The cleaner lazy
shape is an enum over the two source types (`InputFile` / `EncryptedInputFile`,
neither of which is `Clone`) resolved into a cached `(Box<dyn FileRead>, u64)`
— but that's still an extra source enum + `OnceCell` purely to preserve a sync
signature on an API nothing in the crate calls yet. The eager version also
gives us fail-fast on a missing file at construction. I don't think it's worth
the complexity, WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
