adamreeve commented on code in PR #44990:
URL: https://github.com/apache/arrow/pull/44990#discussion_r1955255670
##########
cpp/src/parquet/file_reader.cc:
##########
@@ -314,11 +314,7 @@ class SerializedFile : public ParquetFileReader::Contents {
}
}
- void Close() override {
- if (file_metadata_ && file_metadata_->file_decryptor()) {
- file_metadata_->file_decryptor()->WipeOutDecryptionKeys();
- }
- }
Review Comment:
> That begs the question: when is the InternalFileDecryptor destroyed
exactly?
From debugging some of the dataset tests, it looks like these are cached in
the dataset, as `ParquetFileFragment` holds shared pointer to the
`parquet::FileMetadata` containing the decryptor (although possibly not if
`ScanOptions::cache_metadata` is disabled). So it might not be destroyed until
the whole dataset is destroyed.
> Should we make a copy instead of sharing?
Maybe, although I'm not sure that would work if we re-added this wipe-out on
close or weren't caching the metadata in a dataset. Eg. if you're doing a
dataset scan and create a copy of the properties for each file fragment you
create, the keys could be wiped out after one scan when the file is closed, and
you'd need to somehow recreate a new copy or reinstate the keys if you do
another scan that reuses a previously opened and closed file.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
