mbutrovich commented on issue #2792:
URL: https://github.com/apache/iceberg-rust/issues/2792#issuecomment-5107034008
Noting an interaction with the encryption epic, found while reviewing #2822
(Puffin encryption) for context.
Task 3's loader (implemented locally, not yet a PR) reads the
`deletion-vector-v1` blob via a raw range read straight off `FileIO`, in
`load_deletion_vector` in `caching_delete_file_loader.rs`:
```rust
let blob = basic_delete_file_loader.file_io().new_input(&task.file_path)?
.reader().await?.read(start..start + len).await?;
```
It doesn't check `task.key_metadata` on `FileScanTaskDeleteFile`, even
though that field already exists and the equality/positional-delete branches
right above it in the same function already use it to decrypt Parquet delete
files. So a V3 table combining deletion vectors with table encryption won't
read correctly today. It'll fail loudly, since `DeleteVector::deserialize`'s
length/CRC/magic checks catch the still-encrypted bytes, not silently return
wrong deletes, but the combination is unsupported and nothing currently flags
it.
#2822 validated that `EncryptedInputFile`/`AesGcmFileRead` (already on
`main`, independent of #2822/#2701/#2918 landing) decrypt over
plaintext-relative byte ranges, which is exactly the coordinate space
`content_offset`/`content_size_in_bytes` already use. So the fix for Task 3
doesn't need `PuffinReader::new_from_encrypted`; it can decode
`task.key_metadata` via `StandardKeyMetadata::decode(...)` and wrap the input
in `EncryptedInputFile::new(input, key_metadata)` before the range read, the
same pattern `ManifestFile::load_manifest` already uses
(`crates/iceberg/src/spec/manifest_list/manifest_file.rs:181-199`), which
#2918's manifest-decryption fix depends on.
Scope check: only Task 3 needs to change for this. Task 1 (#2866, blob
codec) has no I/O, so it's unaffected. Task 2 (#2868) already copies
`data_file.key_metadata()` onto `FileScanTaskDeleteFile` unconditionally,
regardless of content type, so DV entries already carry the right key metadata;
that PR needs no change. I'll fold the decrypt step into Task 3 when I open
that PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
